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UNIT I 


Probability and Random Variable 
Probability: 


Set theory 

Experiments 

Sample Spaces, Discrete and Continuous Sample Spaces 
Events 

Probability Definitions and Axioms 
Mathematical Model of Experiments 
Joint Probability 

Conditional Probability 

Total Probability 

Bayes’ Theorem 

Independent Events 

Bernoulli’s trials 


Random Variable: 


e Definition of a Random Variable 
e Conditions for a Function to be a Random Variable 
e Discrete and Continuous, Mixed Random Variable 


UNIT -1 
PROBABILITY AND RANDOM VARIABLE 


PROBABILITY 


Introduction 


It is remarkable that a science which began with the consideration of games of chance 
should have become the most important object of human knowledge. 


A brief history 


Probability has an amazing history. A practical gambling problem faced by the French nobleman 
Chevalier de Méré sparked the idea of probability in the mind of Blaise Pascal (1623-1662), the 
famous French mathematician. Pascal's correspondence with Pierre de Fermat (1601-1665), another 
French Mathematician in the form of seven letters in 1654 is regarded as the genesis of probability. 
Early mathematicians like Jacob Bernoulli (1654-1705), Abraham de Moivre (1667-1754), Thomas 
Bayes (1702-1761) and Pierre Simon De Laplace (1749-1827) contributed to the development of 
probability. Laplace's Theory Analytique des Probabilities gave comprehensive tools to calculate 
probabilities based on the principles of permutations and combinations. Laplace also said, 
"Probability theory is nothing but common sense reduced to calculation." 


Later mathematicians like Chebyshev (1821-1894), Markov (1856-1922), von Mises (1883- 
1953), Norbert Wiener (1894-1964) and Kolmogorov (1903-1987) contributed to new 
developments. Over the last four centuries and a half, probability has grown to be one of the most 
essential mathematical tools applied in diverse fields like economics, commerce, physical 
sciences, biological sciences and engineering. It is particularly important for solving practical 
electrical-engineering problems in communication, signal processing and computers. 
Notwithstanding the above developments, a precise definition of probability eluded the 
mathematicians for centuries. Kolmogorov in 1933 gave the axiomatic definition of probability 
and resolved the problem. 


Randomness arises because of 


> random nature of the generation mechanism 


> Limited understanding of the signal dynamics inherent imprecision in measurement, 
observation, etc. 


For example, thermal noise appearing in an electronic device is generated due to random motion of 
electrons. We have deterministic model for weather prediction; it takes into account of the factors 
affecting weather. We can locally predict the temperature or the rainfall of a place on the basis of 
previous data. Probabilistic models are established from observation of a random phenomenon. 
While probability is concerned with analysis of a random phenomenon, statistics help in building 
such models from data. 


Deterministic versus probabilistic models 


A deterministic model can be used for a physical quantity and the process generating it provided 
sufficient information is available about the initial state and the dynamics of the process generating 
the physical quantity. For example, 


e We can determine the position of a particle moving under a constant force if we know the 
initial position of the particle and the magnitude and the direction of the force. 


e We can determine the current in a circuit consisting of resistance, inductance and 
capacitance for a known voltage source applying Kirchoff's laws. 


Many of the physical quantities are random in the sense that these quantities cannot be predicted 
with certainty and can be described in terms of probabilistic models only. For example, 


e The outcome of the tossing of a coin cannot be predicted with certainty. Thus the 
outcome of tossing a coin is random. 


e The number of ones and zeros in a packet of binary data arriving through a 
communication channel cannot be precisely predicted is random. 


e The ubiquitous noise corrupting the signal during acquisition, storage and transmission 
can be modelled only through statistical analysis. 


How to Interpret Probability 


Mathematically, the probability that an event will occur is expressed as a number between 0 and 1. 
Notationally, the probability of event A is represented by P (A). 


« If P (A) equals zero, event A will almost definitely not occur. 

« If P (A) is close to zero, there is only a small chance that event A will occur. 
= If P (A) equals 0.5, there is a 50-50 chance that event A will occur. 

= If P(A) is close to one, there is a strong chance that event A will occur. 


« If P(A) equals one, event A will almost definitely occur. 


In a statistical experiment, the sum of probabilities for all possible outcomes is equal to one. This 
means, for example, that if an experiment can have three possible outcomes (A, B, and C), then 
P(A) + P(B) + P(C) = 1. 


Applications 
Probability theory is applied in everyday life in risk assessment and in trade on financial markets. 
Governments apply probabilistic methods in environmental regulation, where it is called pathway 


analysis 


Another significant application of probability theory in everyday life is reliability. Many consumer 
products, such as automobiles and consumer electronics, use reliability theory in product design to 
reduce the probability of failure. Failure probability may influence a manufacturer's decisions on a 
product's warranty. 

THE BASIC CONCEPTS OF SET THEORY 


Some of the basic concepts of set theory are: 


Set: A set is a well defined collection of objects. These objects are called elements or members of 
the set. Usually uppercase letters are used to denote sets. 


The set theory was developed by George Cantor in 1845-1918. Today, it is used in almost every 
branch of mathematics and serves as a fundamental part of present-day mathematics. 


In everyday life, we often talk of the collection of objects such as a bunch of keys, flock of birds, 
pack of cards, etc. In mathematics, we come across collections like natural numbers, whole 
numbers, prime and composite numbers. 


We assume that, 

e the word set is synonymous with the word collection, aggregate, class and comprises of elements. 
@ Objects, elements and members of a set are synonymous terms. 

e Sets are usually denoted by capital letters A, B, C, ....., etc. 

e Elements of the set are represented by small letters a, b, c, ....., etc. 


If ‘a’ is an element of set A, then we say that ‘a’ belongs to A. We denote the phrase ‘belongs to’ 
by the Greek symbol ‘€‘ (epsilon). Thus, we say that a € A. 


If ‘b’ is an element which does not belong to A, we represent this as b € A. 
Examples of sets: 
1. Describe the set of vowels. 


If A is the set of vowels, then A could be described as A = {a, e, i, 0, u}. 


2.Describe the set of positive integers. 


Since it would be impossible to list all of the positive integers, we need to use a rule to describe this 
set. We might say A consists of all integers greater than zero. 


3. Set A = {1, 2, 3} and Set B= {3, 2, 1}. Is Set A equal to Set B? 


Yes. Two sets are equal if they have the same elements. The order in which the elements are listed 
does not matter. 


4. What is the set of men with four arms? 


Since all men have two arms at most, the set of men with four arms contains no elements. It is the 
null set (or empty set). 


5. Set A = {1, 2, 3} and Set B= {1, 2, 4, 5, 6}. Is Set A a subset of Set B? 


Set A would be a subset of Set B if every element from Set A were also in Set B. However, this is 
not the case. The number 3 is in Set A, but not in Set B. Therefore, Set A is not a subset of Set B 


Some important sets used in mathematics are 


Z: the set of all integers = {....., -3, -2, -1, 0, 1, 2, 3, ..... } 
Q: the set of all rational numbers 
R: the set of all real numbers 
Z+: the set of all positive integers 
W: the set of all whole numbers 
The different types of sets are explained below with examples. 
1. Empty Set or Null Set: 
A set which does not contain any element is called an empty set, or the null set or the void set and it 
is denoted by @ and is read as phi. In roster form, @ is denoted by {}. An empty set is a finite set, 
since the number of elements in an empty set is finite, i.e., 0. 
For example: (a) the set of whole numbers less than 0. 


(b) Clearly there is no whole number less than 0. 


Therefore, it is an empty set. 


(c)N={x:xEN,3<x <4} 
eLet A= {x:2 <x <3,x is a natural number} 


Here A is an empty set because there is no natural number between 
2 and 3. 


¢ Let B = {x : x is a composite number less than 4}. 
Here B is an empty set because there is no composite number less than 4. 
Note: 

@ + {0} -. has no element. 

{0} is a set which has one element 0. 


The cardinal number of an empty set, i.e., n(@) = 0 


2. Singleton Set: 

A set which contains only one element is called a singleton set. 

For example: 
e A= {x: x is neither prime nor composite} 
It is a singleton set containing one element, i.e., 1. 
¢ B= {x: xis a whole number, x < 1} 
This set contains only one element 0 and is a singleton set. 
eLet A= {x:x € Nand x?=4} 
Here A is a singleton set because there is only one element 2 whose square is 4. 
e Let B = {x : x is a even prime number} 


Here B is a singleton set because there is only one prime number which is even, 1.e., 2. 


3. Finite Set: 


A set which contains a definite number of elements is called a finite set. Empty set is also called a 
finite set. 


For example: 
¢ The set of all colors in the rainbow. 
eN={x:xEN,x<7} 


oP={2, 3,5; 7, 11, 13, 17,2..097} 


4. Infinite Set: 


The set whose elements cannot be listed, i.e., set containing never-ending elements is called an 
infinite set. 


For example: 

¢ Set of all points in a plane 
eA={x:xEN,x>1} 

¢ Set of all prime numbers 
*B={x:xE€W,x=2n} 

Note: 

All infinite sets cannot be expressed in roster form. 
For example: 


The set of real numbers since the elements of this set do not follow any particular pattern. 


5. Cardinal Number of a Set: 


The number of distinct elements in a given set A is called the cardinal number of A. It is denoted 
by n(A). And read as ‘the number of elements of the set’. 


For example: 


eA {x:xEN,x <5} 
A= {1, 2,3, 4} 
Therefore, n(A) = 4 
¢ B = set of letters in the word ALGEBRA 
B= {A,L, G, E, B, R} 
Therefore, n(B) = 6 
6. Equivalent Sets: 


Two sets A and B are said to be equivalent if their cardinal number is same, i.e., n(A) = n(B). The 
symbol for denoting an equivalent set is ‘>’. 


For example: 
A= {1, 2,3} Here n(A) = 3 
B = {p, q, r} Here n(B) = 3 


Therefore, A << B 


7. Equal sets: 


Two sets A and B are said to be equal if they contain the same elements. Every element of A is an 
element of B and every element of B is an element of A. 


For example: 
A= {p, qT, 8} 
B= {p, s.r, q} 
Therefore, A = B 
8. Disjoint Sets: 
Two sets A and B are said to be disjoint, if they do not have any element in common. 


For example; 


A = {x:x is a prime number} 
B = {x : x is a composite number}. 


Clearly, A and B do not have any element in common and are disjoint sets. 


9. Overlapping sets: 
Two sets A and B are said to be overlapping if they contain at least one element in common. 
For example; 
e A= {a, b,c, d} 
B= {a, e, i, 0, u} 
eX={x:xEN,x <4} 
Y={x:xE]-l1<x<4} 
Here, the two sets contain three elements in common, i.e., (1, 2, 3) 
10. Definition of Subset: 


If A and B are two sets, and every element of set A is also an element of set B, then A is called a 
subset of B and we write itas A C Bor BDA 


The symbol C stands for ‘is a subset of or ‘is contained in’ 

¢ Every set is a subset of itself, i.e, AC A,B CB. 

¢ Empty set is a subset of every set. 

* Symbol ‘©’ is used to denote ‘is a subset of’ or ‘is contained in’. 
* A CB means A is a subset of B or A is contained in B. 

* BCA means B contains A. 

Examples; 

1. Let A = {2, 4, 6} 


B = {6, 4, 8, 2} 
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Here A is a subset of B 

Since, all the elements of set A are contained in set B. 
But B is not the subset of A 

Since, all the elements of set B are not contained in set A. 
Notes: 

If ACB and BCA, then A = B, 1.e., they are equal sets. 
Every set is a subset of itself. 


Null set or @ is a subset of every set. 


2. The set N of natural numbers is a subset of the set Z of integers and we write N C Z. 


3. Let A = {2, 4, 6} 
B = {x : x is an even natural number less than 8} 
Here AC Band BCA. 


Hence, we can say A=B 


4, Let A = {1, 2, 3, 4} 
B= {4, 5, 6, 7} 
Here A € B andalsoB ¢€C 


[Z denotes ‘not a subset of” ] 


11. Super Set: 
Whenever a set A is a subset of set B, we say the B is a superset of A and we write, B 2 A. 
Symbol 2 is used to denote ‘is a super set of? 


For example; 
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A= {a,e, 1, 0, u} 


Here A € Biie., A is a subset of B but B 2 Aii.e., B is a super set of A 


12. Proper Subset: 


If A and B are two sets, then A is called the proper subset of B if A € B but B 2 Ae., A#B. The 
symbol ‘C’ is used to denote proper subset. Symbolically, we write A C B. 


For example; 

1. A= {1, 2, 3, 4} 

Here n(A) = 4 

B= {1, 2,3, 4,5} 

Here n(B) = 5 

We observe that, all the elements of A are present in B but the element ‘5’ of B is not present in A. 


So, we say that A is a proper subset of B. 
Symbolically, we write it as A C B 


Notes: 
No set is a proper subset of itself. 


Null set or @ is a proper subset of every set. 


2. A= {p,q,r} 

B= {p,q,r, s, t} 

Here A is a proper subset of B as all the elements of set A are in set B and also A # B. 
Notes: 

No set is a proper subset of itself. 


Empty set is a proper subset of every set. 
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13. Power Set: 


The collection of all subsets of set A is called the power set of A. It is denoted by P(A). In P(A), 
every element is a set. 


For example; 

If A = {p, q} then all the subsets of A will be 
P(A) = {9, {p}, {q}, {p, qh} 

Number of elements of P(A) = n[P(A)] = 4 = 22 


In general, n[P(A)] = 2m where m is the number of elements in set A. 


14.Universal Set 


A set which contains all the elements of other given sets is called a universal set. The symbol for 
denoting a universal set is U or 6. 


For example; 

LIfA={1,2,3} B={2,3,4} C={3,5,7} 
then U = {1, 2,3, 4,5, 7} 

[Here AC U,BGU,C ©Uand UDA, UDB, U DC] 


2. If P is a set of all whole numbers and Q is a set of all negative numbers then the universal set is a 
set of all integers. 


SfA={a,b,cf. B={de} C= {f, 2, h,1} 
then U = {a, b, c, d, e, f, g, h, i} can be taken as universal set. 
Operations on sets: 
1. Definition of Union of Sets: 
Union of two given sets is the smallest set which contains all the elements of both the sets. 


To find the union of two given sets A and B is a set which consists of all the elements of A and all 
the elements of B such that no element is repeated. 
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The symbol for denoting union of sets is ‘VU’. 


Some properties of the operation of union: 


(i) AUB = BUA (Commutative law) 

(ii) AU(BUC) = (AUB)UC (Associative law) 

(iii) AUD=A (Law of identity element, is the identity of U) 
(iv) AUA=A (Idempotent law) 

(v) UUA=U (Law of U) U is the universal set. 

Notes: 


AU @®=@®U A =A {e. union of any set with the empty set is always the set itself. 
Examples: 

1. If A= {1, 3, 7,5} and B = {3, 7, 8, 9}. Find union of two set A and B. 
Solution: 


A UB = {1, 3,5, 7, 8,9} 
No element is repeated in the union of two sets. The common elements 3, 7 are taken only once. 


2. Let X = {a, e, 1, 0, u} and Y = {cp}. Find union of two given sets X and Y. 
Solution: 

XU Y = {a, e, i, 0, u} 

Therefore, union of any set with an empty set is the set itself. 


2. Definition of Intersection of Sets: 


Intersection of two given sets is the largest set which contains all the elements that are common to 


both the sets. 


To find the intersection of two given sets A and B is a set which consists of all the elements which 


are common to both A and B. 
The symbol for denoting intersection of sets is ‘N°. 


Some properties of the operation of intersection 
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Gi) ANB =BNA (Commutative law) 


(ii) (ANB)NC = AN (BNC) (Associative law) 


Gii) DN A=@O (Law of ®) 
(iv) UNA=A (Law of U) 
(v) ANA=A (idempotent law) 


(vi) AN(BUC) = (ANB) U (ANC) (Distributive law) Here M distributes over U 
Also, AU(BNC) = (AUB) MN (AUC) (Distributive law) Here U distributes over N 
Notes: 
AM ®=@0/fA=Q i.e. intersection of any set with the empty set is always the empty set. 
Solved examples : 
1. If A= {2, 4, 6, 8, 10} and B = {1, 3, 8, 4, 6}. Find intersection of two set A and B. 
Solution: 
AN B= {4, 6, 8} 
Therefore, 4, 6 and 8 are the common elements in both the sets. 
2. If X = {a, b, c} and Y = {}. Find intersection of two given sets X and Y. 
Solution: 
XNY={} 
3. Difference of two sets 
If A and B are two sets, then their difference is given by A- BorB- A. 
°If A= {2, 3, 4} and B = {4, 5, 6} 
A - B means elements of A which are not the elements of B. 
1.e., in the above example A - B = {2, 3} 


In general, B- A= {x: x € B, andx € A} 
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¢ If A and B are disjoint sets, then A— B= A and B—- A=B 

Solved examples to find the difference of two sets: 

1. A= {1, 2,3} and B = {4, 5, 6}. 

Find the difference between the two sets: 

(i) Aand B 

(ii) B and A 

Solution: 

The two sets are disjoint as they do not have any elements in common. 
@G@A-B={1,2,3}=A 


(ii) B- A= {4,5,6}=B 


2. Let A = {a, b, c, d, e, f} and B = {b, d, f, g}. 
Find the difference between the two sets: 
(i) Aand B 
(ii) B and A 
Solution: 
(i) A- B= {a,c, e} 
Therefore, the elements a, c, e belong to A but not to B 
(ii) B- A= {g) 
Therefore, the element g belongs to B but not A. 
4. Complement of a Set 


In complement of a set if S be the universal set and A a subset of S then the complement of A is 
the set of all elements of S which are not the elements of A. 


Symbolically, we denote the complement of A with respect to S as A’. 
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Some properties of complement sets 


(i) AU A'=A'UA=U (Complement law) 


(i) (A 1 B')=90 (Complement law) - The set and its complement are disjoint sets. 


(iii) (A U B) = A'N B' (De Morgan’s law) 

(iv) (AN B)'= A' U B' (De Morgan’s law) 

(v) (A')' =A (Law of complementation) 

(vi) ®' = U (Law of empty set - The complement of an empty set is a universal set. 
(vii) U'= ® and universal set) - The complement of a universal set is an empty set. 
For Example; If S = {1, 2, 3, 4, 5, 6, 7} 

A= {1, 3,7} find A’. 

Solution: 

We observe that 2, 4, 5, 6 are the only elements of S which do not belong to A. 
Therefore, A' = {2, 4, 5, 6} 

Algebraic laws on sets: 

1. Commutative Laws: 

For any two finite sets A and B; 

G)AUB=BUA 

Gi) AN B=BNA 

2. Associative Laws: 

For any three finite sets A, B and C; 

Gi) (AU B)UC=AU(BUC) 

(Gi) (AN B)NC=AN(BNC) 


Thus, union and intersection are associative. 
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3. Idempotent Laws: 

For any finite set A; 

G)AUA=A 

G)ANA=A 

4. Distributive Laws: 

For any three finite sets A, B and C; 
G7) AU(BNC)=(AUB)N (AUC) 
Gi) AN (BU C)=(AN B) U(ANC) 
Thus, union and intersection are distributive over intersection and union respectively. 
5. De Morgan’s Laws: 

For any two finite sets A and B; 

(i) A— (BUC) =(A-B)N (A-C) 
(ii) A - (BN C) =(A-B) U(A-C) 
De Morgan’s Laws can also we written as: 
(i) (AUB) =A'NB' 

(11) (AN B)’ =A'UB' 

More laws of algebra of sets: 

6. For any two finite sets A and B; 

(i) A-B=ANB' 

(Gi) B—-A=BNA' 

(ii) A-B=ASANB=@ 

(iv) (A-—B) UB=AUB 


(v) (A-B)NB=@ 
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(vi) (A-— B) U (B— A) =(AUB)-(A NB) 

Definition of De Morgan’s law: 

The complement of the union of two sets is equal to the intersection of their complements and the 
complement of the intersection of two sets is equal to the union of their complements. These are 
called De Morgan’s laws. 

For any two finite sets A and B; 

(i) (A U B)'= A'N B' (which is a De Morgan's law of union). 

(ii) (A 1 B)'= A' U B' (which is a De Morgan's law of intersection). 


Venn Diagrams: 


Pictorial representations of sets represented by closed figures are called set diagrams or Venn 
diagrams. 


Venn diagrams are used to illustrate various operations like union, intersection and difference. 

We can express the relationship among sets through this in a more significant way. 

In this, 

eA rectangle is used to represent a universal set. 

¢ Circles or ovals are used to represent other subsets of the universal set. 

Venn diagrams in different situations 

In these diagrams, the universal set is represented by a rectangular region and its subsets by circles 


inside the rectangle. We represented disjoint set by disjoint circles and intersecting sets by 
intersecting circles. 


S.No Set &Its relation Venn Diagram 


1 Intersection of A and B foe 


id 
fe 
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Union of A and B 


Difference : A-B 


Difference : B-A 


Difference of two sets 


B-A 


Complement of set A 


AUBwhenAcCB 
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A UB when neither A C B 
norBcA 


8 A U B when A and B are 
disjoint sets 

9 (A U B)’ (A union B dash) 

10 (A M B)’ (A intersection B 
dash) 

11 B’ (B dash) 

12 (A - B)’ (Dash of sets A 


minus B) 


13 (A c B)’ (Dash of A subset 
B) 


Problems of set theory: 
1. Let A and B be two finite sets such that n(A) = 20, n(B) = 28 and n(A U B) = 36, find n(A M B). 
Solution: 
Using the formula n(A U B) = n(A) + n(B) - n(A /M B). 
then n(A M B) = n(A) + n(B) - n(A U B) 
= 20 + 28 - 36 
= 48 - 36 


=12 


2. If n(A - B) = 18, n(A U B) = 70 and n(A £O B) = 25, then find n(B). 
Solution: 
Using the formula n(AUB) = n(A - B) + n(A MN B) + n(B - A) 
70 = 18 +25 + n(B - A) 
70 = 43 + n(B - A) 
n(B - A) = 70 - 43 
n(B - A) = 27 
Now n(B) = n(A £O B) + n(B - A) 
= 25 +27 


= 52 
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3. In a group of 60 people, 27 like cold drinks and 42 like hot drinks and each person likes at least 
one of the two drinks. How many like both coffee and tea? 


Solution: 
Let A = Set of people who like cold drinks. 
B = Set of people who like hot drinks. 
Given 
(A U B) = 60 n(A)=27 ~~ n(B) = 42 then; 


n(A M B) =n(A) + n(B) - n(A U B) 


= 27+ 42 - 60 
=69 -60=9 
=9 


Therefore, 9 people like both tea and coffee. 

4. There are 35 students in art class and 57 students in dance class. Find the number of students 
who are either in art class or in dance class. 

¢ When two classes meet at different hours and 12 students are enrolled in both activities. 

e When two classes meet at the same hour. 

Solution: 

n(A)=35, n(B)=57, n(ANB)=12 


(Let A be the set of students in art class. 
B be the set of students in dance class.) 


(1) When 2 classes meet at different hours n(A U B) = n(A) + n(B) - n(A MN B) 


= 35+57-12 
= 92-12 
= 80 
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(11) When two classes meet at the same hour, ANB = @ n (A U B) = n(A) + n(B) - n(A NM B) 
= n(A) + n(B) 
= 35 +57 


= 92 


5. In a group of 100 persons, 72 people can speak English and 43 can speak French. How many can 
speak English only? How many can speak French only and how many can speak both English and 
French? 
Solution: 
Let A be the set of people who speak English. 
B be the set of people who speak French. 
A - B be the set of people who speak English and not French. 
B - A be the set of people who speak French and not English. 
A 1B be the set of people who speak both French and English. 
Given, 
n(A)=72 n(B)=43 = n(AUB)=100 
Now, n(A M B) = n(A) + n(B) - n(A U B) 
= 72 + 43 - 100 
= 115 - 100 
= 15 
Therefore, Number of persons who speak both French and English = 15 
n(A) = n(A - B) + n(A 1 B) 
= n(A - B) = n(A) - n(A /N B) 


= 72-15 
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=) 


and n(B - A) = n(B) - n(A MN B) 


= 43 - 15 


= 28 


Therefore, Number of people speaking English only = 57 


Number of people speaking French only = 28 


Probability Concepts 


Before we give a definition of probability, let us examine the following concepts: 


1. 


Experiment: 


In probability theory, an experiment or trial (see below) is any procedure that can be 
infinitely repeated and has a well-defined set of possible outcomes, known as the sample 
space. An experiment is said to be random if it has more than one possible outcome, 
and deterministic if it has only one. A random experiment that has exactly two (mutually 
exclusive) possible outcomes is known as a Bernoulli trial. 


Random Experiment: 


An experiment is a random experiment if its outcome cannot be predicted precisely. One 
out of a number of outcomes is possible in a random experiment. A single performance of 
the random experiment is called a trial. 


Random experiments are often conducted repeatedly, so that the collective results may be 
subjected to statistical analysis. A fixed number of repetitions of the same experiment can 
be thought of as acomposed experiment, in which case the individual repetitions are 
called trials. For example, if one were to toss the same coin one hundred times and record 
each result, each toss would be considered a trial within the experiment composed of all 
hundred tosses. 


Mathematical description of an experiment: 


A random experiment is described or modeled by a mathematical construct known as a probability 


space. A probability space is constructed and defined with a specific kind of experiment or trial in 


mind. 


A mathematical description of an experiment consists of three parts: 


25 


1. A sample space, Q (or S), which is the set of all possible outcomes. 

2. A set of events , where each event is a set containing zero or more outcomes. 

3. The assignment of probabilities to the events—that is, a function P mapping from events to 
probabilities. 


An outcome is the result of a single execution of the model. Since individual outcomes might be of 
little practical use, more complicated events are used to characterize groups of outcomes. The 
collection of all such events is a sigma-algebra . Finally, there is a need to specify each event's 


likelihood of happening; this is done using the probability measure function,P. 


2. Sample Space: The sample space “is the collection of all possible outcomes of a 


random experiment. The elements of “are called sample points. 
e A sample space may be finite, countably infinite or uncountable. 
e A finite or countably infinite sample space is called a discrete sample space. 
e An uncountable sample space is called a continuous sample space 


Ex:1. For the coin-toss experiment would be the results “Head”’and “Tail”, which we may 
represent by S={H T}. 


Ex. 2. If we toss a die, one sample space or the set of all possible outcomes is 
S={ 1,2, 3,4,:5,6} 

The other sample space can be 

S = {odd, even} 

Types of Sample Space: 

1. Finite/Discrete Sample Space: 

Consider the experiment of tossing a coin twice. 

The sample space can be 


S = {HH, HT, TH , TT} the above sample space has a finite number of sample points. It is 
called a finite sample space. 


2. Countably Infinite Sample Space: 
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Consider that a light bulb is manufactured. It is then tested for its life length by inserting it 
into a socket and the time elapsed (in hours) until it burns out is recorded. Let the measuring 
instrument is capable of recording time to two decimal places, for example 8.32 hours. 
Now, the sample space becomes count ably infinite 1.e. 

S = {0.0, 0.01, 0.02} 
The above sample space is called a countable infinite sample space. 
3. Un Countable/ Infinite Sample Space: 
If the sample space consists of unaccountably infinite number of elements then it is called 
Un Countable/ Infinite Sample Space. 


3. Event: An event is simply a set of possible outcomes. To be more specific, an event is a 
subset A of the sample space S. 


e ACS 
e Fora discrete sample space, all subsets are events. 


Ex: For instance, in the coin-toss experiment the events A={Heads} and B={Tails} would be 
mutually exclusive. 


An event consisting of a single point of the sample space 'S' is called a simple event or elementary 
event. 


Some examples of event sets: 


Example 1: tossing a fair coin 


The possible outcomes are H (head) and T (tail). The associated sample space is SH, 2) It is 
a finite sample space. The events associated with the sample space S are: SAM}.CT} and eo 


Example 2: Throwing a fair die: 
The possible 6 outcomes are: 


(-] Fa Fa Ea) el BB 


ays yf ‘4! 


The associated finite sample space is Sell ee 3.459, 6} .Some events are 
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A=The event of getting an oddface=('l', '3', 'S'}. 
8 = The event of getting a six={'6'} 


And so on. 
Example 3: Tossing a fair coin until a head is obtained 


We may have to toss the coin any number of times before a head is obtained. Thus the possible 
outcomes are: 

H, TH, TTH, TTTH, 
How many outcomes are there? The outcomes are countable but infinite in number. The 
countably infinite sample space is a mda: Tt, Did pax: }. 
Example 4 : Picking a real number at random between -1 and +1 


The associated Sample space is 


S={s|seR,-145<0 =[-1, 1] 


Clearly is a continuous sample space. 

Example 5: Drawing cards 

Drawing 4 cards from a deck: Events include all spades, sum of the 4 cards is (assuming face cards 
have a value of zero), a sequence of integers, a hand with a 2, 3, 4 and 5. There are many more 
events. 

Types of Events: 

1. Exhaustive Events: 


A set of events is said to be exhaustive, if it includes all the possible events. 


Ex. In tossing a coin, the outcome can be either Head or Tail and there is no other possible 
outcome. So, the set of events { H , T } is exhaustive. 


2. Mutually Exclusive Events: 
Two events, A and B are said to be mutually exclusive if they cannot occur together. 


i.e. if the occurrence of one of the events precludes the occurrence of all others, then such a set of 
events is said to be mutually exclusive. 


If two events are mutually exclusive then the probability of either occurring is 
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P(A or B) = P(AU B) = P(A) + P(B). 
Ex. In tossing a die, both head and tail cannot happen at the same time. 


3.Equally Likely Events: 

If one of the events cannot be expected to happen in preference to another, then such events 
are said to be Equally Likely Events.( Or) Each outcome of the random experiment has an 
equal chance of occuring. 

Ex. In tossing a coin, the coming of the head or the tail is equally likely. 

4. Independent Events: 


Two events are said to be independent, if happening or failure of one does not affect the happening 
or failure of the other. Otherwise, the events are said to be dependent. 


If two events, A and B are independent then the joint probability is 
P(A and B) = P(AN B) = P(A)P(B), 


5.Non-. Mutually Exclusive Events: 


If the events are not mutually exclusive then 


P(Aor B) = P(A)+P(B) —P(Aand B). 


Probability Definitions and Axioms: 
1. Relative frequency Definition: 
Consider that an experiment E is repeated n times, and let A and B be two events associated 
withE. Let ng and ng be the number of times that the event A and the event B occurred 
among the n repetitions respectively. 
The relative frequency of the event A in the 'n' repetitions of E is defined as 
f(A) = na /n 
The Relative frequency has the following properties: 
1.0 <f(A) <1 


2. f(A) =1 if and only if A occurs every time among the n repetitions. 
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If an experiment is repeated ” times under similar conditions and the event 4 occurs in 


"4 times, then the probability of the event A is defined as 


PCa) fin 
Limitation: 
Since we can never repeat an experiment or process indefinitely, we can never know the probability 
of any event from the relative frequency definition. In many cases we can't even obtain a long 
series of repetitions due to time, cost, or other limitations. For example, the probability of rain 
today can't really be obtained by the relative frequency definition since today can’t be repeated 
again. 

2. .The classical definition: 


Let the sample space (denoted by 8 ) be the set of all possible distinct outcomes to an 
experiment. The probability of some event is 


number of ways the event can occur 


number of outcomes in $ 


s 


provided all points in“ are equally likely. For example, when a die is rolled the probability of 
1 


getting a 2 is 6 because one of the six faces is a 2. 
Limitation: 


What does "equally likely" mean? This appears to use the concept of probability while trying to 
define it! We could remove the phrase "provided all outcomes are equally likely", but then the 


definition would clearly be unusable in many settings where the outcomes in S did not tend to 
occur equally often. 


Example1:A fair die is rolled once. What is the probability of getting a ‘6’ ? 


Hare AN — {1 * eae ‘Si 4 5! '6"} aaa A ={ '5 ‘} 


N=6 and N,=1 
wl 
“P(A = 3 


Example2:A fair coin is tossed twice. What is the probability of getting two ‘heads'? 


Here 97 (HH, TH, HT, TT) gq A= (HH) 


Total number of outcomes is 4 and all four outcomes are equally likely. 


Only outcome favourable to 4is {HH} 
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Probability axioms: 


Given an event £ in a sample space § which is either finite with ¥ elements or countably infinite 
with ¥ = elements, then we can write 


s=(U4) 


and a quantity P (i), called the probability of event #i, is defined such that 


Axiom1: The probability of any event A is positive or zero. Namely P(A)=0. The probability 
measures, in a certain way, the difficulty of event A happening: the smaller the probability, the 
more difficult it is to happen. i.e 


O<P(%)<1 


Axiom2: The probability of the sure event is 1. Namely P(Q)=1. And so, the probability is always 
greater than 0 and smaller than 1: probability zero means that there is no possibility for it to happen 
(it is an impossible event), and probability 1 means that it will always happen (it is a sure event).i.e 


P(Sy=1. 


Axiom3: The probability of the union of any set of two by two incompatible events is the sum of 
the probabilities of the events. That is, if we have, for example, events A,B,C, and these are two by 
two incompatible, then P(AUBUC)=P(A)+P(B)+P(C). i.e Additivity: 


P(E, U Ex)=P (,)+P (22), where £1 and © are mutually exclusive. 


P (Ua) 2) = Xi P(E) for n= 1, 2, .... where £1, #2, ... are mutually exclusive 
(i.e., 2 1B: =), 


Main properties of probability: If A is any event of sample space S then 


P(A)+P(A)=1. Or P(A)=1-P(A) 

Since AUA=S, P(A UA)=1 

The probability of the impossible event is 0, i.e P(O)=0 

If ACB, then P(A)<P(B). 

If A and B are two incompatible events, and therefore, P-A—B)=P(A)—P(ANB).and 
P(B-A)=P(B)—P(ANB). 

Addition Law of probability: 


ik oe ba 


a 


dl 


P(AUB)=P(A)+P(B)-P(ANB) 


Rules of Probability: 


Rule of Subtraction: 


Rule of Multiplication: 


Rule of addition: 


PERMUTATIONS and COMBINATIONS: 


S.No. | PERMUTATIONS COMBINATIONS: 

1 Arrangement of things in a specified In permutations, the order of arrangement of 
order is called permutation. Here all objects is important. But, in combinations, 
things are taken at a time order is not important, but only selection of 

objects. 

2 Arrangement of ‘r’ things taken at a 


time from ‘n’ things ,;where r < nina 
specified order in called r-permutation. 


3 Consider the letters a,b andc . 
Considering all the three letters at a 
time, the possible permutations are 
ABC ,acb,bca,bac,cbaandcab 


4 The number of permutations taking r The number of combinations taking r things at 


things at a time from ‘n’ available a time from ‘n’ available things is denoted as 
things is denoted as p(n,r) ornp, C(n,r)ornC, 
5 nP.= r!/nC= n!/(n-r)! nCr=P(n,r)/r!=n!/r!(n-r)! 


Example 1: An urn contains 6 red balls, 5 green balls and 4 blue balls. 9 balls were picked at 
random from the urn without replacement. What is the probability that out of the balls 4 are red, 3 
are green and 2 are blue? 

Sol: 


af OE 13t 
9 balls can be picked from a population of 15 balls in. F161, 
AT ay x 5c x 4 
4 3 = 
Cy 


2 


Therefore the required probability is 
Example2: What is the probability that in a throw of 12 dice each face occurs twice. 


Solution: The total number of elements in the sample space of the outcomes of a single 
throw of 12 dice is = 64 


The number of favourable outcomes is the number of ways in which 12 dice can be 
arranged in six groups of size 2 each — group | consisting of two dice each showing 1, group 2 
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consisting of two dice each showing 2 and so on. 
Therefore, the total number distinct groups 


_ 12 
21212121212! 


_ 12! 
6 gla 
Hence the required probability is (2)'6 


Conditional probability 


The answer is the conditional probability of B given A denoted by PBI A) We shall 
develop the concept of the conditional probability and explain under what condition this 


conditional probability is same as PB) ' 


Notation 


= Conditional probability of B 


Let us consider the case of equiprobable events discussed earlier. Let Nas sample points be 
favourable for the joint event ANB 


Figure 1 


15052 | aan 


Number of outcomes in A 


n( AB) 


nA) 2A PLA) 


% 
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This concept suggests us to define conditional probability. The probability of an event B under the 
condition that another event A has occurred is called the conditional probability of B given A and 
defined by 


P(ANB) 


P(BI A) = , P(A) #0 


We can similarly define the conditional probability of A given B , denoted by PAIS) : 


From the definition of conditional probability, we have the joint probability PLANS) of 


two events A and B as follows 


P(AMB) = P(A)P(B I A) = P(B) P(A! B) 


Problems: 
Example 1 Consider the example tossing the fair die. Suppose 


A=event of getting an evennumber = (2,4, 6} 
8 =event of getting anumber less than 4 = {1, 2, 3} 
Ad = {2} 

_P(ANB) _ 1/6 _ 1 


a emormarT a 


Example 2 A family has two children. It is known that at least one of the children is a girl. What is 
the 
probability that both the children are girls? 


A = event of at least one girl 


B = event of two girls 


S = (gg, gb, bg, bb}, A={(gg, gb, bg} and B = {gg} 
ANB = (gg) 
_P(ANB)_ 1/41 


TO eee ae 
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Properties of Conditional probability: 
Lvs A ‘hen P(B!A)=1and P(A! 8) 2 PCA) 


We have, ANS =8 


P(ANB) _ PA) _, 


| PUBTA) = P(A) P(A 
and 
P(AIB) == = ) 
_ P(A)P(BI A) 
P(B) 
_ PA) 
PB) 
> P(A) 
2. Since P(AMB) 20, P(A) >0 
“P(BIA)= a >0 
3. We have, BU Hatt A) PA) Ly 
P(A P(A) 


4. Chain Rule of Probability/Multiplication theorem: 


POA Ay...) = P(A) PCA! A PCA IA O.4)..PCA FA OY... 1A) 


We have, 


(ANB NC) =(ANDOC 
PANS NC) = PANSP(CIANS) 
=PAP(BIAP(CI ANB) 
UL PRANB OC) = PAP(BIAP(CIANS) 


We can generalize the above to get the chain rule of probability for n events as 


POA Ay...) = POA) PCA! A IPA IA O.A)..PCA EA Oy. AL) 
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Joint probability 


Joint probability is defined as the probability of both A and B taking place, and is 
denoted by P (AB) or P(ANB) 


Joint probability is not the same as conditional probability, though the two concepts are 
often confused. Conditional probability assumes that one event has taken place or will take place, 
and then asks for the probability of the other (A, given B). Joint probability does not have such 
conditions; it simply asks for the chances of both happening (A and B). In a problem, to help 
distinguish between the two, look for qualifiers that one event is conditional on the other 
(conditional) or whether they will happen concurrently (joint). 


Probability definitions can find their way into CFA exam questions. Naturally, there may 
also be questions that test the ability to calculate joint probabilities. Such computations require 
use of the multiplication rule, which states that the joint probability of A and B is the product of 
the conditional probability of A given B, times the probability of B. In probability notation: 


P(AB) = P(A |B) * P(B) 


Given a conditional probability P(A | B) = 40%, and a probability of B = 60%, the joint 
probability P(AB) = 0.6*0.4 or 24%, found by applying the multiplication rule. 


P(AUB)=P(A)+P(B)-P(AnB) 
For independent events: P(AB) = P(A) * P(B) 


Moreover, the rule generalizes for more than two events provided they are all independent of one 


another, so the joint probability of three events P(ABC) = P(A) * (P(B) *P(C), again assuming 
independence. 


Summary of probabilities 
Event Probability 
aA | P(A)€ (0,1) 


nottA | P(A‘) =1-— P(A) 
) 


nore P(AUB) = P(A) + P(B) — P(ANB) 
P(AUB)=P(A)+P(B) _ ifAandB are mutually exclusive 
aandg P(A B) = P(A|B)P(B) = P(B\A)P(A) 


P(AN B) = P(A)P(B) if A and B are independent 


P(ANB P(B\A)P(A 
A given B P(A|B)= => - a 
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Total Probability theorem: 


Let Ay Ay... Arie n events such that 


=Awvu A... U4 and 4 _=@ fori = j. 
Bein A Siac ? | Then for any event B, 


P(B) => P(A)P(BIA) 
i-l 


Proof: We have 


“ P(B) = P(JBNA) 


i-1 


=>P(B NA) 
i-] 


= 5 P(A)P(BI A) 
i-] 


Figure 3 


Remark 


(1) A decomposition of a set S into 2 or more disjoint nonempty subsets is called a partition of 


S.The subsets 4>4)> - - -» 4y form a partition of S if 
S=AU A VA and AA = ¢ fori * j. 


(2) The theorem of total probability can be used to determine the probability of a complex event 
in terms of related simpler events. This result will be used in Bays' theorem to be discussed to 
the end of the lecture. 
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Bayes' Theorem: 


Suppose the event 2 occurs if one of the events A,4.... Ay occurs. Thus we have the information of the 
probabilities P(A) and P(B/A),i =1,2..,2, We ask the following question: 

Given that B has occured what is the probability that a particular event A, has accured? In other words 
what is P(A, 1B)? 


We have P(B)= > PA) P(B|A) (Using the theorem of total probability) 


im] 
P(A.) P( BLA) 
PB) 

__ P(A) P(B/A,) 


> PA JP(BI A) 
im] 


PA, |B) = 


This result is known as the Baye's theorem. The probability PA? is called the a priori probability and 
P(A, 1B) is called the a posteriori probability. Thus the Bays' theorem enables us 


to determine the a posteriori probability PA, 1B) from the observation that B has occurred. This 
result is of practical importance and is the heart of Baysean classification, Baysean estimation etc. 


Examplel1: 

In a binary communication system a zero and a one is transmitted with probability 0.6 and 0.4 
respectively. Due to error in the communication system a zero becomes a one with a probability 
0.1 and a one becomes a zero with a probability 0.08. Determine the probability (1) of receiving a 
one and (ii) that a one was transmitted when the received message is one 


Solution: 
Let S is the sample space corresponding to binary communication. Suppose th be event of 


Transmitting 0 and Tbe the event of transmitting 1 and % and *' be corresponding events of 
receiving 0 and | respectively. 
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Given 7(%)= 9.6. PCR) = 0.4, PRI) = 0.1, 4 PR! %) = 0.08. 


(1) P(A) = Probabilty of receiving ‘one’ 

= PR) (RIT) + PPR LT) 
= 0.4«0.92+0.6x0.1 
= 0.448 

(1) Using the Baye's rule 

PR) P(R IT) 

P(R) 
os POPE) 
PUL)P(RIT) + (GPR IG) 
7 0.4x 0.92 
0.4x0.92+0.6x0.1 
= 0.8214 


PIR) = 


Example 7: In an electronics laboratory, there are identically looking capacitors of three makes 


0 
Ais, and As. the ratio 2:3:4. It is known that 1% of 4 1.5% of 4 and-2%.of fy are defective. 
What percentages of capacitors in the laboratory are defective? If a capacitor picked at defective 


is found to be defective, what is the probability it is of make 4 ? 
Let D be the event that the item is defective. Here we have to find PD) and PCA, 1D) : 


2 1 4 
P(A) = 5, P(A.) == and P(A) => 
HOA) = UU, WA) = UID and P(D/ A) = 0.02 


“. P(D) = P(A) P(D! A) + P(A) P(D! A.) + P(A) P(DI A) 
= 2x9 o14+4x0.0154+4 x0 02 
9 3 9 
=0.0167 
and 
_ PAP! A) 
MAID) 


4 0.02 
9 


0.0167 
= 0.533 


A0) 


Independent events 


Two events are called independent if the probability of occurrence of one event does not 
affect the probability of occurrence of the other. Thus the events A and B are independent if 


P(BIA)=P(B)., P(AIB)=P(A) 


and 


PA) ang PA 


where and are assumed to be non-zero. 


Equivalently if A and B are independent, we have 


P(AMB) 


PA) PB) 


P(A mB) = P(A)P(B) Joint probability is the 


or product of individual 


probabilities. 


Two events A and B are called statistically dependent if they are not independent. Similarly, we 


+ Ay 


can define the independence of n events. The events Ai 
only if 


are called independent if and 


P(A A.) = P(A) PCA,) 
PLA A, A.) = PLA)PCA) PCA) 
PUA A, OA, 7...A) = P(A) PCA) PCA)..P(A) 


Example: Consider the example of tossing a fair coin twice. The resulting sample space is 
given by 8 ={#4,4T,TH,TT} and all the outcomes are equiprobable. 


Let A={1Hi,TT} be the event of getting ‘tail’ in the first toss and B= (1H, HH} be the 
event of getting ‘head' in the second toss. Then 


P(A)= P(B)= 


oe os 


1 
2 and 


(A mB) “hits so that 


Again, 
P(AnB)- - - P(A)P(B) 


Hence the events A and B are independent. 
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Problems: 


Examplel.A dice of six faces is tailored so that the probability of getting every face is 
proportional to the number depicted on it. 


a) What is the probability of extracting a 6? 

In this case, we say that the probability of each face turning up is not the same, therefore we cannot 
simply apply the rule of Laplace. If we follow the statement, it says that the probability of each face 
turning up is proportional to the number of the face itself, and this means that, if we say that the 
probability of face 1 being turned up is k which we do not know, then: 

PCL} )=k, PU2})=2k, PU3 })=3k, PU 4 ))=4k, 

P({5})=5k,P({ 6})=6k. 

Now, since {1},{2},{3},{4},{5},{6} form an events complete system , necessarily 

PULL} APC 2})+PU3 +P 4) )+PC5 +P 6} =1 


Therefore 


k4+2k4+3k+4k+5k+6k=1 


which is an equation that we can already solve: 
21k=1 


thus 
k=1/21 


And so, the probability of extracting 6 is P({6})=6k=6-(1/21)=6/21. 
b) What is the probability of extracting an odd number? 


The cases favourable to event A= "to extract an odd number" are: {1},{3},{5}. Therefore, since 
they are incompatible events, 
P(A)=P({1})+P({3})4+P({5 })=k+3k4+5k=9k=9- (1/2 1)=9/21 
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Example2: Roll a red die and a green die. Find the probability the total is 5. 


Solution: Let %”? represent getting * on the red die and ” on the green die. 


S={ 1) C2) (3) + (1,6) 
(2,1) (2,2) (2,3) + (2,6) 
(3,1) (3,2) (3,3) « (3,6) 


(6,1) (6,2) (6,3) +++ (6,6)} 
Then, with these as simple events, the sample space is 
The sample points giving a total of 5 are (1,4) (2,3) (3,2), and (4,1). 
4 
Therefore P (total is 5) = 36 


Example3: Suppose the 2 dice were now identical red dice. Find the probability the total is 5. 


Solution : Since we can no longer distinguish between (2.9) and 9%) , the only distinguishable 
points in S are 


S={ C1) C2) (3) + (6) 
(2,2) (2,3) + (2,6) 
(3,3) «+ (3,6) 


(6, 6)} 


Using this sample space, we get a total of > from points (1,4) and (2, 3) only. If we assign equal 
A 2 
probability 21 to each point (simple event) then we get P (total is 5) = 21. 


Example4: Draw 1 card from a standard well-shuffled deck (13 cards of each of 4 suits - 

spades, hearts, diamonds, and clubs). Find the probability the card is a club. 

Solution 1: Let * = { spade, heart, diamond, club}. (The points of 5 are generally listed between 
1s 

brackets {}.) Then S has 4 points, with 1 of them being "club", so e (club) = 4. 

Solution 2: Let * = {each of the 52 cards}. Then 13 of the 52 cards are clubs, so 


P(club) = & = +. 
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Example 5: Suppose we draw a card from a deck of playing cards. What is the probability 
that we draw a spade? 


Solution: The sample space of this experiment consists of 52 cards, and the probability of each 
sample point is 1/52. Since there are 13 spades in the deck, the probability of drawing a spade is 


P(Spade) = (13)(1/52) = 1/4 


Example 6: Suppose a coin is flipped 3 times. What is the probability of getting two tails and 
one head? 


Solution: For this experiment, the sample space consists of 8 sample points. 
S = {TTT, TTH, THT, THH, HTT, HTH, HHT, HHH} 
Each sample point is equally likely to occur, so the probability of getting any particular sample 
point is 1/8. The event "getting two tails and one head" consists of the following subset of the 
sample space. 
A = {TTH, THT, HTT} 
The probability of Event A is the sum of the probabilities of the sample points in A. Therefore, 
P(A) = 1/8 + 1/8 + 1/8 = 3/8 
Example7: An urn contains 6 red marbles and 4 black marbles. Two marbles are 
drawn without replacement from the urn. What is the probability that both of the marbles are 


black? 


Solution: Let A = the event that the first marble is black; and let B = the event that the second 
marble is black. We know the following: 


« In the beginning, there are 10 marbles in the urn, 4 of which are black. Therefore, P(A) = 


4/10. 
« After the first selection, there are 9 marbles in the urn, 3 of which are black. Therefore, 
P(BJA) = 3/9. 


Therefore, based on the rule of multiplication: 


P(A NB) = P(A) P(BJA) 
P(A NB) = (4/10) * (3/9) = 12/90 = 2/15 
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RANDOM VARIABLE 


INTRODUCTION 

In many situations, we are interested in numbers associated with the outcomes of a random 
experiment. In application of probabilities, we are often concerned with numerical values which are 
random in nature. For example, we may consider the number of customers arriving at a service 
station at a particular interval of time or the transmission time of a message in a communication 
system. These random quantities may be considered as real-valued function on the sample space. 
Such a real-valued function is called real random variable and plays an important role in describing 
random data. We shall introduce the concept of random variables in the following sections. 


Random Variable Definition 


A random variable is a function that maps outcomes of a random experiment to real 
numbers. (or) 
A random variable associates the points in the sample space with real numbers 


A (real-valued) random variable, often denoted by X(or some other capital letter), is a function 
mapping a probability space (S; P) into the real line R. This is shown in Figure 1.Associated with 
each point s in the domain S the function X assigns one and only one value X(s) in the range R. 
(The set of possible values of X(s) is usually a proper subset of the real line; i.e., not all real 
numbers need occur. If S is a finite set with m elements, then X(s) can assume at most an m 
different value as s varies in S.) 


A random variable: a function 


X 


Domain: probability space Range: real line 
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Examplel 

A fair coin is tossed 6 times. The number of heads that come up is an example of a random 
variable. 

HHTTHT — 3, THHTTT -- 2. 

These random variables can only take values between 0 and 6. The 

Set of possible values of random variables is known as its Range. 

Example2 

A box of 6 eggs is rejected once it contains one or more broken eggs. If we examine 10 boxes of 
eggs and define the randomvariablesX1, X2 as 

1 X1- the number of broken eggs in the 10 boxes 

2 X2- the number of boxes rejected 

Then the range of X1 is {0, 1,2,3,4-------------- 60} and X2 is {0,1,2 -- --- 10} 

Figure 2: A (real-valued) function of a random variable is itself a random variable, 1.e., a 
function mapping a probability space into the real line. 


Example 3 Consider the example of tossing a fair coin twice. The sample space is S={ 
HH,HT,TH,TT} and all four outcomes are equally likely. Then we can define a random variable 
& as follows 


Sample Point Value of the 
random 


Variable 


Here Re he : 


Example 4 Consider the sample space associated with the single toss of a fair die. The 
sample space is given by * ={1,2,3,4,5,6} . 


If we define the random variable * that associates a real number equal to the number on 
the face of the die, then * = t!:2.3,4,5,6} | 


Types of random variables: 
There are two types of random variables, discrete and continuous. 


1. Discrete random variable: 
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A discrete random variable is one which may take on only a countable number of distinct 


values such as 0, 1,2,3,4,........ Discrete random variables are usually (but not necessarily) counts. If 
a random variable can take only a finite number of distinct values, then it must be discrete 
(Or) 


A random variable * is called a discrete random variable if Fz () is plece-wise constant. 


Thus vee is flat except at the points of jump discontinuity. If the sample space Sis discrete the 
random variable * defined on it is always discrete. 


F(x) 


Oo x 


Plot of Distribution function of discrete random variable 


°A discrete random variable has a finite number of possible values or an infinite sequence of 
countable real numbers. 

—X: number of hits when trying 20 free throws. 

—X: number of customers who arrive at the bank from 8:30 — 9:30AM Mon--Fri. 

—E.g. Binomial, Poisson... 


2. Continuous random variable: 


A continuous random variable is one which takes an infinite number of possible values. Continuous randot 
variables are usually measurements. E 
A continuous random variable takes all values in an interval of real numbers. 


(or) 
: : , : fx. : F 
X is 72-4 a continuous random variable if wz ;, ,1 absolutely continuous function 


ee) ; : a 
of x . Thus “***’ is continuous everywhere on R and ** “’ exists everywhere except at finite or 
countably infinite points 


3. Mixed random variable: 


_X is called a mixed random variable if " (x) has jump discontinuity at countable number of 
points and increases continuously at least in one interval of X. For a such type RV X. 


AT 


Plot of Distribution function of continuous and mixed random variables 
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UNIT I 
Distribution and density functions and Operations on One Random Variable 


Distribution and density functions: 


e Distribution function and its Properties 

e Density function and its Properties 

e Important Types of Distribution and density functions 
> Binomial, Poisson 

> Uniform 

> Exponential 

> Gaussian 

> Rayleigh 

e Conditional Distribution 

e Methods of defining Conditioning Event 

e Conditional Density function and its properties 
e Problems 


Operation on One Random Variable: 


e Expected value of a random variable, function of a random variable 

e Moments about the origin 

e Central moments - variance and skew 

e Characteristic function 

e Moment generating function 

e Transformations of a random variable 
> Monotonic transformations for a continuous random variable 
> Non monotonic transformations of continuous random variable 
> Transformations of Discrete random variable 


UNIT-II 
DISTRIBUTION AND DENSITY FUNCTIONS AND OPERATIONS ON ONE RANDOM 
VARIABLE 


Probability Distribution 


The probability distribution of a discrete random variable is a list of probabilities associated with 
each of its possible values. It is also sometimes called the probability function or the probability 
mass function. 


More formally, the probability distribution of a discrete random variable X is a function which 
gives the probability p(xi) that the random variable equals xi, for each value xi: 
p(xi) = P(X=xi) 


It satisfies the following conditions: 


Cumulative Distribution Function 


All random variables (discrete and continuous) have a cumulative distribution function. It is a 
function giving the probability that the random variable X is less than or equal to x, for every value 
Xs 


Formally, the cumulative distribution function F(x) is defined to be: 
P(x) = PU 4 x) 

for 
=o 4x < @ 


For a discrete random variable, the cumulative distribution function is found by summing up the 
probabilities as in the example below. 


For a continuous random variable, the cumulative distribution function is the integral of its 
probability density function. 


Example 
Discrete case : Suppose a random variable X has the following probability distribution p(x1): 


xi =O OCU 3 4 5 

p(xi) 1/32 5/32 10/32 10/32 5/32 1/32 
This is actually a binomial distribution: Bi(5, 0.5) or B(S, 0.5). The cumulative distribution function 
F(x) is then: 

xi Oo 41 2 3 4 5 
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F(xi) 1/32 6/32 16/32 26/32 31/32 32/32 


F(x) does not change at intermediate values. For example: 
F(1.3) = F(1) = 6/32 and F(2.86) = F(2) = 16/32 
Probability Distribution Function 


The probability PUK S$ x}) = P{s| X(s) $x, s€ Sis called the probability distribution 
function (also called the cumulative distribution function , abbreviated as CDF ) of * and 


denoted by ge . Thus 
By (a) = POL § x}) 


Value ofthe random variable 


Random variable 


Properties of the Distribution Function 


1 OS Fy(x) 41 


2. Fy () is a non-decreasing function of “ . Thus, if ays agp tenes aa 
Ky SX 
=> (X(s) <x} C{X) S my) 
=> PLX(s) Sm} S PLAX (s) < xy} 
Wy Byly)< Fy ly) 
O #(@) Is right continuous. 
F(x") =lim Fy (xh) = Fy() 


Aro 


Because, lima fy (x th) = lira PLA (sy xt hh 


Aro io 
=P{Ats) & x} 
=F (*) 

fy (—o7).=0 
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Because, #,(-©) = P{s| X(s) <-~} = P(g) = 0 


Py(2) =1 
Pim < XS yp) =F,0.)- Fy)! Se} = PES) =1 


We have, 


F(x) = Pe 
&y0 
= lim P(X(s) £ x-h} 
6. PULX > a)= P(x <X <o})=1- F(x) 
= P{X(s) < x} - P(X (s) =x) 
=F, (x) - P(X =x) 


Example: Consider the random variable in the above example. We have 
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Value of the 
random PUX =x}) 
Variable X =x 


= — 


For x <0, 
Fk = Pepe £-2tj = 0 


Por ll ga <1, 
Fy(a) = PUK $2) = PUK =) = 


Por 14 x< 2, 
By (x) = PULX <x} 

= POX =O} U(X = 9) 

= PAX =0/)+ PUY =) 


For 22x43, 
By (ay = PUA S33} 
= PUA =wWLY =] UL =24) 
ara sO PP = ot Pe we 
Ld 1.8 
SH es 
4 4 4 4 
For x 2 3, 
Py (x) = PULL § x}) 
= PS) 
=] 
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For 2£x< 3, 
Fy (x) = POX £33) 
=PUL-2 0 UL =} or = 2) 
= PUX = 0}) + POX =) + PUK = 23) 
1 3 
=.+ —a 
4 4 
Thus we have seen that given Fy (QQ), co< x <00 , we can determine the probability of any event 
involving values of the random variable * .Thus FyQ) WR EX ig g complete description of the 


random variable * . 
Example 5 Consider the random variable defined by 
Fy (x) = 0, 5 Goat 


peters =2 S240 
4 


=1 x20 
Find a) 26 =9)_ 
by P(X $0} 
PAX? 
a PCRS 
Solutio a) P(X = 0) = F, (0°) - Fy (0°) 
ae 
ete 
4 4 


b) P{X <0} = Fy (0) 


c) P(X > 2} =1-F,(2) 
=1-1=0 
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Probability Density Function 


The probability density function of a continuous random variable is a function which can be 
integrated to obtain the probability that the random variable takes a value in a given interval. 


More formally, the probability density function, f(x), of a continuous random variable X is the 
derivative of the cumulative distribution function F(x): 
d 
f(x) = — F (x) 
dx 
F(x) = 2S x) it follows that: 
Jax) ax = F(b\ - F(a) = Pla < X< 3d) 


Since 


If f(x) is a probability density function then it must obey two conditions: 


a. that the total probability for all possible values of the continuous random variable X is 1: 
fax)ax = 1 


b. that the probability density function can never be negative: f(x) > 0 for all x. 
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Example 1 


Consider the random variable “ with the distribution function 


0 x<0 
1 
q 04x61 
ps aa 
= 18x<2 
2 
i 222 


The plot of the Fy () is shown in Figure 7 on next page. 


The probability mass function of the random variable is given by 


Value of the random 


variable X =x Px(x) 
0 

1 
4 
1 1 
4 

2 _ 
1 
2 
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Properties of the Probability Density Function 


w------ This follows from the fact that Fy) is a non-decreasing function 


F(3)= f fx Wau 


[ fe(adar=1 


® 


4. P(xn<XSx)= f fax 


-% 


Other Distribution and density functions of Random variable: 


1. Bernoulli random variable: 


Suppose X is a random variable that takes two values 0 and 1, with probability mass 
functions 


py) = P(X =} =p 


Px(9) =1-p, O< ps 


And 


Such a random variable X is called a Bernoulli random variable, because it describes the 
outcomes of a Bernoulli trial. 


The typical CDF of the Bernoulli RV “is as shown in Figure 2 


By (x) 
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Mean and variance of the Bernoulli random: 


by =BX => ky (k) =1xp+0x(-p) =p 
ae 
1 
EX? ="D'k’py(k) = 1X p+ 0X(1- p) = p 
40 


“0% = BX? ~ yy = p(l- p) 


Remark 


e The Bernoulli RV is the simplest discrete RV. It can be used as the building block for 
many discrete RVs. 
e For the Bernoulli RV, 


EX™ =p m=1,2,3... 
Thus all the moments of the Bernoulli RV have the same value of ”” 


2. Binomial random variable 


Suppose X is a discrete random variable taking values from the set (O.1,....7) . 4 is called a 


binomial random variable with parameters n and OS<pSl if 


py) ="Cyp*d- py *  k=0,1..,” 
where 


~ n! 


Ck ki(n—k)! 


The trials must meet the following requirements: 


the total number of trials is fixed in advance; 

there are just two outcomes of each trial; success and failure; 
the outcomes of all the trials are statistically independent; 

all the trials have the same probability of success. 


ao Tf 


As we have seen, the probability of k successes in n independent repetitions of the Bernoulli 
trial is given by the binomial law. If X is a discrete random variable representing the number of 
successes in this case, then X is a binomial random variable. For example, the number of heads in ‘n 
"independent tossing of a fair coin is a binomial random variable. 


A ~ BU, p); 


e The notation is used to represent a binomial RV with the parameters * and 
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PF 

Py = ge iO Pp =|ptll =p =, 
Spr = Cyr py =[p +0 py = 
i) , i) 


The sum of n independent identically distributed Bernoulli random variables is a 


binomial random variable. 
The binomial distribution is useful when there are two types of objects - good, bad; 


correct, erroneous; healthy, diseased etc 


Example1:In a binary communication system, the probability of bit error is 0.01. If a block of 8 bits 
are transmitted, find the probability that 


(a) Exactly 2 bit errors will occur 
(b) At least 2 bit errors will occur 
(c) More than 2 bit errors will occur 


(d) All the bits will be erroneous 


Suppose * is the random variable representing the number of bit errors in a block of 8 bits. 
Then A ~ B(8, 0.01). 


Therefore, 


(a) Probability that exactly 2 bit errors will occur 
= px) 
= *c, x0.017 x0.99° 
= 0.0026 
(b) Probability that at least 2 bit errors will occur 
= px(Q)+ py) t Px 
= 0.99" + 8c x0.01 x0.997 + ®c, x0.017 x0.99° 


=, 2900 


(c) Probability that more than 2 bit errors will occur 
2 
=1- 2 px(k) 
k=0 
=1-0.9999 
=0.0001 


(@}) Probability that all 8 bits will be erroneous 
= py (8) 
elie str 
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|p cal 


Bionomial Distribution with p = 0.8 .n=6 


o3s 


The probability mass function for a binomial random variable with n = 6 and p =0.8 
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Mean and Variance of the Binomial Random Variable 


We have 
BX =p, (b) 
t=O 


= "SEC ph - py 
kal 
=0xg" oe" -py'* 


“Sk Te ao? ml eer 


kml 


= xk 
errr Was pi? @ P) 


: aa ll —e 

aia =» 
rat CK — Tilt —) 
<= 2a 


=a ia -g” 


k=0 


“d-py*" (Substituting & =k-1) 


=np(p +1- py 
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Sumilarly 
BX? = Sk’ py(®) 
inl 
“oF Or dpe 
=0?xg" +> k*C,p*(1-p)** 
kel 
ni 


" 2)" Fie-B 


i=l 


p*(i-p)** 


* nl hie ane 
2 Diem? OP 
n-ll pa 7 pyre 


~ wp > (k-l+ DO 


. -1 . n-il 
- k=] : bly — py) yy — py He) 
wr e-piasi-kepe +2 G-Diesizeepr? OP 


=mpX(n-l)p + np 
=n(n-1)p? + np 


Where 


a (2-1)! Relig yy t1{h-}) 
20 VG@a-epr Oo?) 


1s the mean of B(-1,p) 
.. Oy = variance of X 
= n(n - Dp? + mp -n?p? 
=xp(1-p) 
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3. Poisson Random Variable 


A discrete random variable X is called a Poisson random variable with the parameter 4 if 4 > 9 
and 


Px(k)= (e*A*)/k! 


The plot of the pmf of the Poisson RV is shown 


04--— Y r oe v “rT T 
= 
A=1 
S02 ~ 
& > 
0.1 
0 2 -9.6-0.-.0-0-5-0-6-8-6--6-0-6-4 
0 2 4 6 8 10 12 14 16 8 2 
h 
125 1 
v T Y } 
Y | 
o 7 
| 
=z 0.15 4 is | 
» ¥ 
Pins 1 
| 
0.05 
7 on | 
f ‘oe —_t_f 2 6. 6.6. 65 6-6-6 6-6-D 
0 2 4 is 5 10 12 14 16 128 20 


. 
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Mean and Variance of the Poisson RV 


The mean of the Poisson RV Xis given by 


Hy = Sex (x) 
kmO 


BX? =D py (®) 


Lf} 
© -A ak 
-0+ SS ~ 
fal k! 
. ae. 
tak 1! 
ag SED 1+1)A* 
ta OK 1! 


Pa 
a 2! — 1! 


gee ee 
fa'k - 2! tak 


= 274 Ate4 +24 Ae? 
=+A 
Oy = BX? - ph =A 


Example:The number of calls received in a telephone exchange follows a Poisson distribution 
with an average of 10 calls per minute. What is the probability that in one-minute duration? 


1. no call is received 
i. exactly 5 calls are received 
iil. More than 3 calls are received. 
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Solution: Let X be the random variable representing the number of calls received. Given 
Where + =!°. Therefore, 


- ae SURE 
i. probability that no call is received ~?x)=#” = 9.000095 


-10 495 
e” x10 
ii. probability that exactly 5 calls are received 3! 0.0378 
iii. _ probability that more the 3 calls are received 


5 2 3 
“1- Spy t-e Mae P+ 4S 
k=O ! 


Poisson Approximation of the Binomial Random Variable 


The Poisson distribution is also used to approximate the binomial distribution B(x, p) when n is 
very large and p is small. 


Consider binomial RV with © ~ 4?) with 
no, p> 0 so that BX =xp =A remains constant. 


Then 


Py’) =" Pp - py 


2! & nk 
i 1- 
l= Bi” (1- p) 
_ 2% — I) 2) (2 -k+1) 


& 1- n-k 
xi p (- p) 


| #a-a-2) , te 
a Zi ia 


_ ~ 3 Pd = py 


k-1) 
2 (mp)*(1- p)** 


a-24-5..a- 
iin 3 Pé3 
kl 
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a 


a” (1-—)(1-=)...d- ) ‘ : 
= ”% 4] 3 - »\*" 
Se (l- p) 


Lore So np BHU 
==). nd — ) 
— %_ (mp)*(1- py** 


2 Bn ge A 
(1 -ha -=)...-——)(ay*"a- =)" 
a Se i, Pe: ae 


7 


A 
kia -=)* 
” 
? ee 
Note that lim(1——)* =e 
»o a 


tees 9 ee 
(-—)0-=)...d-——(4y*- =)" age 
O.Dy(k) =lim —“2—__4___#______#4 _=__— 


xia - 4 el 
” 


Thus the Poisson approximation can be used to compute binomial probabilities for large n. It also 
makes the analysis of such probabilities easier. Typical examples are: 


e number of bit errors in a received binary data file 


e Number of typographical errors in a printed page 


Example 4 Suppose there is an error probability of 0.01 per word in typing. What is the probability 
that there will be more than | error in a page of 120 words? 


Solution: Suppose X is the RV representing the number of errors per page of 120 words. 


AX ~B120,2) where P = 9-91- Therefore, 


| A=120x0.01= 0.12 
P(more than one errors) 
=1-py(0)- px 
= |-e" — Ae” 
= 0.0066 


In the following we shall discuss some important continuous random variables. 
4. Uniform Random Variable 


A continuous random variable X is called uniformly distributed over the interval [a, b], 
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-o <a@<h<o, ifits probability density function is given by 


1 


a axsb 
ty =4B—a sii 
0 otherwise 

tx{x) 


a bX 


Figure 1 


We use the notation & ~U(@, 8) to denote a random variable X uniformly distributed over the 
interval [a,b] 


Distribution function By () 


For x< a 
Fy (x) =0 
Fora ix seh 


[few 
“| 


aa 


ete 
-a 


For x >, 
Fy (x)= 1 


Figure 2 illustrates the CDF of a uniform random variable. 
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a bh x 


Figure 2: CDF of a uniform random variable 


Mean and Variance of a Uniform Random Variable: 


© ’ - 
Mz = BX = f xfy(3)dx = | rae 


_atb 


2 


2 


2 4 
x 
EX? = f* Sn (X)dx = —* 
~ <a 


_b* +ab+a? 
3 
oh Ex? - 2-2 tabta’ (ard) 
= . 3 4 
(b- a)’ 
12 
The characteristic function of the random variable 4 ~U(a, a) is given by 


3 Jr 
gy (w) = He™ = Jax 
a¥Y-@ 


2 — owe 


- jw(b-a) 
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5. Normal or Gaussian Random Variable 


The normal distribution is the most important distribution used to model natural and man made 
phenomena. Particularly, when the random variable is the result of the addition of large number of 
independent random variables, it can be modeled as a normal random variable. 


continuous random variable X is called a normal or a Gaussian random variable with 
parameters “xand “x if its probability density function is given by, 


2 
nea 


1 
1 - ay 
Ip() "= : 
. Wem y -w <x <e0 


>0 


Where Hx and ox are real numbers. 


N ree 
We wie thal ig (My Oy ae, 


: oo 
ip Hx ~ 0 and as 1 and the random variable X is called the standard normal variable. 


1 = 
2 


Fy(X) ™ Jor 


Figure 3 illustrates two normal variables with the same mean but different variances. 


[\ 
} 


i sigma 
; | 


ff j \ \sigma=2 
/ i \ 


Figure 3 
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. oz Is a bell-shaped function, symmetrical about aa? 


2 2 
« © Determines the spread of the random variable X . If “x is small X is more 


concentrated around the mean Hf x 


e Distribution function of a Gaussian variable is 


Fy (x) = P(X <x) 


x ftom 
= 1 fe ry ) dt 
Vota, + 
ae | 
Substituting oe , we get 
ty 
| oy 1, 
F(x) =—a= ff oe? aes 
7 af 297 J 
eth] 
ay 
where P(x) is the distribution function of the standard normal variable. 


Thus Fy () can be computed from tabulated values of O() . The table P(x) was very useful in the 


pre-computer days. 


In communication engineering, it is customary to work with the Q function defined by, 


Q(x) =1- (x) 
a 1 " ee 
= ire 2 du 
00-4, 9r-x)-9) 
Note that z and 
Q(x) =1- g(-x) 


These results follow from the symmetry of the Gaussian pdf. The function Q@) is tabulated and the 
tabulated results are used to compute probability involving the Gaussian random variable. 
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Using the Error Function to compute Probabilities for Gaussian Random Variables 


The function OQ) is closely related to the error function ery (x) and the complementary error 
function erfe(x) ‘ 


erf (x) = hen 


Note that, 


And the complementary error function erfe(x) is given 


> = 


erfe(x) = —<= fe du 
fe ( x) Jat 


=1-erf(x) 
hs 


~ 3 
f° 2 du 
x 


eames 
. O(x) z= 


I il 
NIE wl 
—— 4 

= “ 

> 

s— 

4s 4], 
—.. nN] ° 
2 yoo 
NS} 
nd 
— 


Mean and Variance of a Gaussian Random Variable 


N 2 
xis M Aer Oe eee then 
BA = iy 


war(X) = Oy" 


Proof: 
_ 4a 
EX = taf tpn — fx 1] dx 
[Pee Fam 4 
i lia 
= 2. [ (way + Hx) e 2 Tyla 
or S 
| wo ity to iy 
= lus + ag 
rd eed hl 
z 
7 1 ¢r 
“OFM I cfts - 
3, il Substituting Hx =i 
_ Hy MP ie oz 
= Site ae & 
fom {? ae so that x= udy + My 
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Var(X) = B(X- u,)° 
1 

/270y 
2-3" ital 5" 


- = fox*e @ 7 oydu (substituting u = “2 
gay 
X -—. 


‘ 2 2) 

[(x-uyye her? ax 

6 a 

Ox 
= 2x Se ate Ea 


o.? Bt : 7 al 
= 2x 2 [f7e° at bstituting £ = — 
or f e (subs ing > 


2 
= ox %& rT 3 
Va (2 
2 
= 2x 2x s"(5] 
2 2 
2 
wn 


= r. 


6. Exponential Random Variable 


A continuous random variable * is called exponentially distributed with the parameter 


Ae *™ >0 
(0)=| . 


0, otherwise 


A > 0 if the probability density function is of the 
PDF of Exponential Random Variable is 


Example 1 
Suppose the waiting time of packets in “ina computer network is an exponential RV with 


F(x) = 0.589" x20 
Then, 


05 
P({0.1< X <0.5}) = [052° "ax 
ol 


=? 5x05 _ goo 


= 0.0241 
Rayleigh Random Variable 
A Rayleigh random variable X is characterized by the PDF 


et = 2 
sg ¥ ide : x20 


Jy(Q= x 
0. x<0 


where “is the parameter of the random variable. 


probability density functions for the Rayleigh RVs are illustrated in Figure 


Figure 6 


(2 


Mean and Variance of the Rayleigh Distribution 


similarly 
EX? = [ 7 f,@)dk 
eal 
0 a 
—_ a # 
= 207 [ue du (Substituting « = —~) 
rf Za 


=20° ( Noting that [uen*che is the mean of the exponential RV with i=1) 
0 
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Relation between the Rayleigh Distribution and the Gaussian distribution 


A Rayleigh RV is related to Gaussian RVs as follow: If %~NO) ang 


- {ya 2 
4, ~N(QO,a°) are independent, then the envelope sili i Saar has the Rayleigh 
distribution with the parameter ® . 


We shall prove this result in a later lecture. This important result also suggests the 
cases where the Rayleigh RV can be used. 


Application of the Rayleigh RV 
v 


Modeling the root mean square error- 


Modeling the envelope of a signal with two orthogonal components as in the case of a signal 
of the following form: 


Conditional Distribution and Density functions 


We discussed conditional probability in an earlier lecture. For two events A and B 


with PLB) * 0 , the conditional probability P(A! B) was defined as 
P(ANMB) 
P(A/ 8) =—_— 
P(8) 


Clearly, the conditional probability can be defined on events involving a Random Variable X 


Conditional distribution function: 


< 
Consider the event (x & x} and any event B involving the random variable X . The 
conditional distribution function of X given B is defined as 


Fy (xi B)= P[(X <x} /B] 
_ P(X sa} 08] 


ee P(B)=0 


We can verify that AGB 


Particularly. 


) satisfies all the properties of the distribution function. 


Fy(-0/B) =O.) Fy(@/B)=1 


0< Fy (x/B)<1 
Fy (xi B) 


Is a non-decreasing function of *. 
i. Pifag <0 <x,}/B) = P(X S$ x,}/ B)- PUX Sx} / 8) 
= By (0! B)- By (mi 8) 
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Conditional Probability Density Function 


In a similar manner, we can define the conditional density function Sx (xf B) of the 
random variable X given the event B as 


Si (x18) 5, (xB) 


All the properties of the pdf applies to the conditional pdf and we can easily show that 
Fz (xf B)20 


[fe (x! Bde = Fy(#/B)=1 


: By (xi B)= ffx (@/ 2 Ja 


0 


P({ x < XS x,}/B) = Fy (x, / B)- Fy (x,/ 8) 


. [fe (x/ Bax 


Example 1 Suppose X is a random variable with the distribution function se (x) . Define 
B= {X S b} 


Case 1: * <2 


Then 


6) 


And 


Case 2: *22 


Fy(xtB) gfx (2/8) 


are plotted in the following figures. 


Figure 1 
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Fy xl B) 


(ee 


Example2: Suppose * is a random variable with the distribution function oe (x) and 


B=(X >d} 


Then 
< = 
For 4 28 14 Sx} W(X > 8) ? Therefore, 


F, (xi B)=0 xib 


Forx > b (X Sxpr(X ob} =(b< Xx} therstore: 


Fy, (x/B)= ee 


Thus, 


the corresponding pdf is given by 
0 


fy (x! B) = fe (x) 


xb 


otherwise 


xb 


otherwise 
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OPERATION ON RANDOM VARIABLE-EXPECTATIONS 


Expected Value of a Random Variable 


The expectation operation extracts a few parameters of a random variable and provides 
a summary description of the random variable in terms of these parameters. 


It is far easier to estimate these parameters from data than to estimate the distribution or 
density function of the random variable. 
Moments are some important parameters obtained through the expectation operation. 


Expected value or mean of a random variable 


The expected value of a random variable * is defined by 


EX = { xfy(xdx 


f fax 


Provided -» exists. 
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EX Js also called the mean or statistical average of the random variable ¥ and is denoted by 
Hy. 


N. 


Note that, for a discrete RV * with the probability mass function (pmf) Pz(H) PL, the 


pdf Fz) is given by 


pie 
Fy (2) = > py (4) d(x - x) 


i=l 
“My = BLY] = 9 xB pa (a) (@- x) 


= x(x) f x8(x- x) ax 


=Papxa) vv | x6(x-x)dx 


0 


Thus for a discrete random variable XY with ?% (%),8=1,2,....,M, 


iv 
Hy = ee RP yl) 
jel 


Figurel Mean of a random variable 
Example 1 


Suppose ~ is a random variable defined by the pdf 
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1 


Fel = 4b - 
0, 


aixsb 


otherwise 


Hy = J fglrx 
mfx : ax 
2 b-@ 
_a@t+b 
2 


Example 2 


Consider the random variable * with the pmf as tabulated below 


Value of the random 
variable x 


px(x) 


N 
Hy “2 %Px(%) 


sift aesBhetos Sesto ate 
8 8 4 2 


St 
3 


Example 3 Let X be a continuous random variable with 


a 
Real ) a(x? +a?) 
Then 


EBX = (ae (x)ax 


“ 
aa 
— [as 
_ EX +e 
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ot 7 
—In(l+x } 


Hence EX does not exist. This density function is known as the Cauchy density function. 


Expected value of a function of a random variable 


Suppose ¥=g9(*) is a real-valued function of a random variable * as discussed in the last 


class. 
Then, 


BY = Bg(X)= | ata) fy(addsx 


We shall illustrate the above result in the special case ® (*) when » ~~ & (x) is one-to-one and 
monotonically increasing function of x In this case, 


Figure 2 


o2 


Jylx) 


gt) ree) 


Fyly) = 


EY = [fe ney 


a -1 
_ yrs ON » 
3 gle Oo) 


where y, = g(-™) andj, = g(@). 
Substituting x = g7(v) so that y = g(x) and dy = g'(xdx, we get 


HY =| g(x) fy (ajax 


The following important properties of the expectation operation can be immediately 
derived: 


(a) If © is a constant, 2° =¢ 


Eo= { cfy(ndx=cf fp(ddkac 
Clearly “© “© 


(b) If g,(X) and g,(*) are two functions of the random variable ¥ and “! and c, are 


constants, 
Ele, 8 (4) + €y8,( 40 oo, Bg (4Y) + c, 2,4, 


Elegy 4) + egg, (4) = i Glee) + yey) Fy ae 
=fog@s@art | qe, fend 


=o, | a(Oh@dx+e, | gO f(Odx 
=o42) (4+ e, Eg, (4) 


The above property means that #is a linear operator. 
MOMENTS ABOUT THE ORIGIN: 


] " 
Mean-square value ay? FP feladae 
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MOMENTS ABOUT THE MEAN 
Variance 


Second central moment is called as variance 


For a random variable ¥ with the pdf x) and mean #x: the variance of ¥ is denoted by ox 
and 


o2 = BX — py) = | (x- py) fy (Bax 


defined as 


Thus for a discrete random variable ¥ with ?% (4), = 1,2,.....4, 


OF =5 (% - Hy) Py (4) 


2 aKa 
The standard deviation of ¥ is defined as °* ~ HCL Hy) 


Example 4 


Find the variance of the random variable in the above example 


oF = BX - py 
3 ath, 1 
Sea ae 
1 


3 
= pa gllx*ak- 2x 


a+b? 
2 


_@-ay 
42 


Example 5 


Find the variance of the random variable discussed in above example. As already computed 
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ot = EX A, 
7 
5 


1 


its.4 ie. fs. 
={0- x—+(1- Ko+¢2- xI+(3- 
3? gt 3° a *' a? ra 


yx 
meu 

64 
X, andX 


For example, consider two random variables 2 with pmf as shown below. Note that 


2 _9 


x, 


X, and X 


each of 2has zero mean.The variances are given by 3 implying that 


x, has more spread about the mean. 


Properties of variance 
(1) O% = EX? - wy 
ox = E(X~ px) 
= E(X? -2uyX + 22) 
= EX*-2y, 8X + Ent 


= EX? - 2y5 + uy 
= EX’= 2 


0 Oy) = EX? by! 
(2) If Y=cX +b, where c and are constants, 4 oF =c* 04 


of =E(eX +b-cpy -bY 


= Ec*(X - By Y 


2-2 


(3) If “is a constant, 


nth moment of a random variable 


We can define the nth moment and the nth central- moment of a random variable X by the 
following relations 
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nth-order moment #Y™ = i x fe(xjdx n=l, 2,.. 


nth-order central moment #(Y - wy, )* = i (x py)” fe (dx n= 1, 2, ... 
Note that am 


EX 


= ‘ 2 ‘a: 
The mean “£ is the first moment and the mean-square value 4%” is the 


second moment 


aS. ” 2 
The first central moment is 0 and the variance 7% * E(X — py) is the second 
central moment 


SKEWNESS 


e The third central moment measures lack of symmetry of the pdf of a random variable 
E(X - py y 


3 
oe is called the coefficient of skewness and if the pdf is symmetric 


this coefficient will be zero. 

The fourth central moment measures flatness or peakedness of the pdf of a random 
E(X - py - 

variable. oY Is called kurtosis. If the peak of the pdf is sharper, then the 

random variable has a higher kurtosis. 


Characteristic function 
Consider a random variable with probability density function Fz@) The characteristic 
function of * denoted by G;(@), is defined as 
gy (@) = Eel 
= f get Jy (xjdx 
where fF 


Note the following: 


° 0; (2), is a complex quantity, representing the Fourier transform of Fx) and 


as : Jak , jax. : j ' 
traditionally using ® instead of # This implies that the properties of the Fourier 
transform applies to the characteristic function. 


We can get 


Fx@) from 9; (2), by the inverse transform 
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Example 1 
Consider the random variable X with pdf Fx@) given by 


asxsb 
= 0 otherwise. The characteristics function is given by 


$e(@) = (oe -) 
j@ 


(@-2) 


Solution: 


Example 2 
The characteristic function of the random variable * with 


fy(x)=ae™ 2A>0,x>0is 


dg(@) = Je" de™ 


= ay gl ae 
0 


A 
AW- jw 


Characteristic function of a discrete random variable 


Suppose X is a random variable taking values from the discrete set a { 


corresponding probability mass function Px (% ) for the value *? 


Then, 
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@y(@) = Hel** 
= oe Px (3; jeleh 


If Rx is the set of integers, we can write 


‘ ‘ : ; , : jax 
In this case Oy (@), can be interpreted as the discrete-time Fourier transform with ® 


substituting # vin the original discrete-time Fourier transform. The inverse relation is 


1 a - 
py(k)= eal? ik gp (@)d@ 


py(k)=p-p)*, k=0,1,...,is given by 
bz(@) = > e?*p(l-p)* 
af} 
=p Se" (1- py 
kf 
1=(1- pel 
Moments and the characteristic function 


Given the characteristics function x (@), the nth moment is given b 
g y 


i ae 
9) det 


al 
F ; F F jad 
To prove this consider the power series expansion of # 


Saye yr! et eats 
etait jox+lOe, Gere 
! ft! 


EX, BX? EX" 


Taking expectation of both sides and assuming to exist, we get 


Pee 2 - om x 
(jo) BX “e , U9) BX ‘ 


Gy (@) = 1+ joRX + 
2! nl 
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Taking the first derivative of 9; (2), with respect to Wat @= Owe get 


de 28 
ada lio 


Similarly, taking the nth derivative of 9; (2), with respect to Wat @= Owe get 


Thus , 


EX = 1dg@) 
j da 


a0 
and generally 


axe = 1 br) 
Je aa” 


a0 


TRANSFORMATION OF A RANDOM VARIABLE 


Description: 


Suppose we are given a random variable X with density fX(x). We apply a function g 
to produce a random variable Y = g(X). We can think of X as the input to a black 
box,and Y the output. 
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UNIT Hl 


Multiple Random Variables and Operations on Multiple Random Variables 


Multiple Random Variables: 


Vector Random Variables 

Joint Distribution Function and Properties 

Joint density Function and Properties 

Marginal Distribution and density Functions 

Conditional Distribution and density Functions 

Statistical Independence 

Distribution and density functions of Sum of Two Random Variables and Sum of Several 
Random Variables 

Central Limit Theorem 


Operations on Multiple Random Variables: 


Expected Value of a Function of Random Variables 

Joint Moments about the Origin 

Joint Central Moments 

Joint Characteristic Functions 

Jointly Gaussian Random Variables: Two Random Variables case and N Random 
Variable case, Properties 

Transformations of Multiple Random Variables 


UNIT-3 
MULTIPLE RANDOM VARIABLES 


Multiple Random Variables 


In many applications we have to deal with more than two random variables. For example, 
in the navigation problem, the position of a space craft is represented by three random variables 
denoting the x, y and z coordinates. The noise affecting the R, G, B channels of colour video 
may be represented by three random variables. In such situations, it is convenient to define the 
vector-valued random variables where each component of the vector is a random variable. 


In this lecture, we extend the concepts of joint random variables to the case of multiple 
random variables. A generalized analysis will be presented for *? random variables defined on the 
same sample space. 


Jointly Distributed Random Variables 


We may define two or more random variables on the same sample space. Let “ and ¥ be 


two real random variables defined on the same probability space (SF, ?). The mapping Sor 


se, (X(s),¥() eR? 


such that for is called a joint random variable. 


X(s).Y(s) 


Figure 1 


Joint Probability Distribution Function 


Recall the definition of the distribution of a single random variable. The event {4 $3} was 


used to define the probability distribution function By () . Given Te , we can find 
the 
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probability of any event involving the random variable. Similarly, for two random variables “ 
and J , the event {A Sx,P Sym {HK SaENY Sy} is considered as the representative event. 

The probability PU $%.F <y}) V@,y)eR? 
joint cumulative distribution function (CDF) of the random variables * and ¥ and denoted by 
yy (x,y) 


is called the joint distribution function or the 


(x,y) 


Figure 2 
Properties of JPDF 


yyy) satisfies the following properties: 


1) By y (4M) S Pyy Oy, yt 4 Sx andy, Sy, 


If xm <x, and yy, < y9, 

{X £4, ¥ Sy} CLK Sx,F¥ < yy} 
PLA Rag ek Sy SE Rony Pek yy} 
By yy) Fy G2) 

3) Py y(-™, y) a Py y(x,-@) =0 


2) 


Note that “ <~*.-¥ Sy} C{4 S$ -2} 
4) Fy y(9, 09) = | 
5) Fray) is right continuous in both the variables. 


6) If x < x, and y,<), 


Pm <A 8%, ¥, SF Sy} Py (%Q.¥4) — Py y (2) oa Py y(%.¥,) + Py (4. ¥) 
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Given "RRs age ,we have a complete description 


of the random variables * and f . 


7) Py (x) = Py (x, +09) 


To prove this 
{X Sx} H={X LAMY S to} 
| Aya) = P({X S xh) = P(X £2,¥ £ &}) = Fy (x, +2) 


Similarly 202 =F Oy... 


eas es en , each of F(x) and #0) 5. called a marginal 


Given 


Distribution function or marginal cumulative distribution function (CDF). 


Jointly Distributed Discrete Random Variables 


If X and ¥ are two discrete random variables defined on the same probability space 


(5,2, 2) such that -* takes values from the countable subset Ry and ¥ takes values from the 
countable subset Ry .Then the joint random variable (X,Y) can take values from the countable 
subset in gy . The joint random variable (4,9) is completely specified by their joint 
probability mass function 


Pyy(y) = Pis|X@)= 24) =y}, Vaye Ry Ry 


Given ?* x(%y) , we can determine other probabilities involving the random variables * and 


¥ 


Remark 


. Pxy(ay) =0 for (x,y) ¢ Ry x Ry 


2 Pyy(,y) mil 
o CADW Ry Ry 
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> Peyliwv=PC U _{x.y}) 


Cay Je Ry Sy (xP )eRyx Re 


=PUR, XR) 
=Pis|(4(s), Fise (Ry * Ry} 
This is because =P(S) =1 


¢ Marginal Probability Mass Functions: The probability mass functions Px(2) and ” yO) 
are obtained from the joint probability mass function as follows 
Px(x) = PUX = x}URy) 
=> Pxy(%y) 
ped 


and similarly 


Py(y)= 2 Pyy(%y) 


These probability mass functions Px(2) and ?¥ 0) obtained from the joint probability mass 
functions are called marginal probability mass functions . 
Example 4 Consider the random variables “ and / with the joint probability mass function as 


tabulated in Table 1. The marginal probabilities Px(3) and ” r0) are as shown in the last column 
and the last row respectively. 


estate 


Joint Probability Density Function 


If * and ¥ are two continuous random variables and their joint distribution function is 


continuous in both* and ” ne we can define joint probability density function Sxyy) by 


Jyyls, y= gy (% ¥) 
a provided it exists. 
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x »} 
Py y (x,y) = J f Jxxluiv)dvdu 
Clearly 3o eo 


Properties of Joint Probability Density Function 


» Fxr@y) is always a non-negative quantity. That is, 


fey(%y20 Vix,%eR? 


J Jxy (x, y)dxdy = 1 


¢ The probability of any Borel set can be obtained by 


PB)= fl fey, y)dxdy 
(x, pRB 
Marginal density functions 


The marginal density functions Fx) and Jy) of two joint RVs * and are given 
by the derivatives of the corresponding marginal distribution functions. Thus 


Fx@=§Fx@) 
Fy (x,0) 


a 
ak 
a 
a 


=2 5 (f feyu rade 


= | ferayry 
KeO= | fey vay 
Thus F=f fey vey 


and similarly $()= f fey(myax 


Jey (x,y) 


Example 5 The joint density function of the random variables in Example 3 is 
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Z 


tap *? (x) 


yy (Gy = 


(-e*)0-e")] #2 0,p20 
acy 


= 2a 7g x20y»20 


Example 6 The joint pdf of two random variables # and ¥ are given by 


Fuylaylscay OSxS2, 08 yS2 
=0 otherwise 
* Find °. 
¢ Find By y(, ¥) 
¢ Find Fy) and AO) 


* What is the probability “0 $# £49< FS) 9 


(Cf ter orev =e ff aoa 


x 
2 
x 
Ps 


similarly 


=% 
AON 5 
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POO <.¥ <40<F 2h 


ey a0 te (O,0) es FCO, 1 2 FP, 0,05 
+ +0-0-0 
16 
1 


16 


Conditional Distributions 


We discussed the conditional CDF and conditional PDF of a random variable conditioned on 
some events defined in terms of the same random variable. We observed that 


P((X <a} mB) 


Fy (af B= PUB) 


P(B\*0 


fy (at B) =F, (x12) 


We can define these quantities for two random variables. We start with the conditional 
probability mass functions for two random variables. 


Conditional Probability Density Functions 


Suppose “ and ¥ are two discrete jointly random variable with the joint PMF Pxx(%y). The 
conditional PMF of ¥ given * = *is denoted by Prix!) and defined as 


PyrgQvix) = POP = ALA = xh) 
fs =a} OE oH) 
PX =H 
= Pxx(%y) provided py(x) #0 
Py() 


This 


7 


Pyyl*,¥) 
Pypyly! x) ee 


provided py(x) =O 
Py(X) 


Similarly we can define the conditional probability mass function 


Puy (x,¥) 


provided py(3) = 0 
Py) 


Puylal y¥) = 
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Conditional Probability Distribution Function 


Consider two continuous jointly random variables * and / with the joint probability 


distribution function  * ay) We are interested to find the conditional distribution function of 
one of the random variables on the condition of a particular value of the other random variable. 


We cannot define the conditional distribution function of the random variable Y on the 


condition of the event ‘ = *} by the relation 
F (yfxH=PY Ss yiX =x) 
ix 


_PWsy,X =3) 
P(X =x) 


as P(4=%)=0 in the above expression. The conditional distribution function is defined in the 


limiting sense as_ follows: 


Fo (yixj=fmy i PY Sylxe X Sx+ Ax) 
Vix 


PUY Sy, x ¢X Sx+ Ax) 


=linn 
m0 P(x XSx4+AR) 


I ey (10) Ard 


=| ; had 
BPE ie a) z (cuAx 


I Fey (aude 


—w 


ar ars 


¥ 
J Sey (7, w)etta 


<P Ola=2 ae 


Conditional Probability Density Function 


Frix VIX =) = fx! X); 


is called the conditional probability density function of ¥ 
given x 


Let us define the conditional distribution function . 


The conditional density is defined in the limiting sense as follows 
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Frege =x) slimy, yyy ly + Al = 2) — yy (A = ayy fay 
Fp Ol A= 2) = 1M psoas Oe Poel as A Sat by By (yl es A oath Ay 


Because, (X =x) slimy 9 (a < XS x+Ax) 


The right hand side of the highlighted equation is 


lity soaxso rex + Ayla co xt Any Fy yy Cel x a ox + Ax) Ay 
= lity, soarsolPly <P S yt Api x c¥ S x+ Ax))/ Ay 
= him yy so urs0l Ply cP S yt Aya cd Sxt Anyi Pla< XY Sx+ AnjAy 
=limy, so urso Sey Os YAxdy! fy CAxAy 
= Fey (YM fel) 


. dx(yla= Fay (a, yu Ata) 


Similarly we have 


Fey laxly)= fey yi Ay) 


2 
Two random variables are statistically independent if for all (ayeR’, 


FrixQlay= fy) 
or equivalently 


Fery)= frre) 


Example 2 X and Y are two jointly random variables with the joint pdf given by 


Jey(sy) =& for0£x51 


= Oothenwise 


find, 


(a) * 
(bfx) and fy Ov) 


(pfx y) 


Solution: 
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f [Fee rear =1 
Since 


We get 
1 
A x—xlxl=1 
2 
=>k=2 
“i dyy(Q.y)=2 forOSxSlasySx 


= OD otherwise 


Fy) = f fx2@. ray _ ha = 2x 


© 1 
ful) = f fay (a. y)dx = 2fdx = 20-y) 
= 4 


Independent Random Variables (or) Statistical Independence 


Let * and ¥ be two random variables characterized by the joint distribution function 
Py y (xy) = PUX £x,F Sy} 


= 3 
x,y) = Paks, 
and the corresponding joint density function Fxg y) Gedy * XY (x, ¥) 


a 
Then “ and ¥ are independent if Vix, yE RR’, (4 $3} and (FS y} 


Thus, 


are independent events. 
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Py y (ayy = PLA SXF Sy} 
=PLX &£ x} P(F 4 ¥} 
= Fy Cy 0) 
PP ey (x, ¥) 
axdy 
3 Rela) GE) 
ax ay 
= f(A) 
a Juylay) = Fy Ay OY} 


oo Fg Cy) = 


and equivalently Jig) = FO) 


Sum of Two Random Variables 


We are often interested in finding out the probability density function of a function of two or 
more RVs. Following are a few examples. 


¢ The received signal by a communication receiver is given by 


Ze A+Y 


where ~ is received signal which is the superposition of the message signal * and the noise f . 


x (> Zz 


hi 


* The frequently applied operations on communication signals like modulation, 
demodulation, correlation etc. involve multiplication of two signals in the form Z = XY. 


We have to know about the probability distribution of Zin any analysis of Z . More formally, 
I. XY (x,y) 


given two random variables X and Y with joint probability density function and a 


Z=¢(X.Y) 


*we have to find Sz (z) . 


function 
In this lecture, we shall address this problem. 


Probability Density of the Function of Two Random Variables 
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=) 
We consider the transformation & ROR 


< 
Consider the event (2 <2} corresponding to each z. We can find a variable subset 


Dy oR oon that Dz “(rl ear) < z} 


Figure 1 


1, Fy (z) = PZ <2) 
= P{(xy)|(%»)€ By} 
| j Fay (ay) dydx 


(ry 


and f, (-)- 2) 


Probability density function of Z=X+Y. 


Consider Figure 2 
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Figure 2 


We have 


TALS 


>X+Y¥ Sz 


Therefore, see is the colored region in the Figure 2. 


“Ay (z)= li Fzy (x,y )dxdy 


(oF 


5 f The | 


“| 


~ J f fxx (a> sa substituting » =u -x 


- f f tex (a= Ha] du interchanging the order of integration 


a z m 
Aglz)= Zz if f ter Lawae - a di 


= (le (x,.u-x)dx 


ae = Lee (x,u—x)dx 
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If X and Y are independent 
Fay (e2—7)= Fy (2) A (27-2) 
“tel2)= f fel A (e- 2) 
= Fa l2)* Fy (2) 
Where * is the convolution operation. 
Example 1 


Suppose X and Y are independent random variables and each uniformly distributed over (a, b). 
tz (%) And ty (Y) are as shown in the figure below. 


fx(x) 
1/b-a 


The PDF of 42 = 4 + is a triangular probability density function as shown in the figure. 


Central Limit Theorem 


Se Smee 4 


Consider ” independent random variables » .The mean and variance of 


each of the random variables are assumed to be known. Suppose BUA) = Hy, and 
X,) = oF 
sale, . Form a random variable 


Yo =X,+X,+..4X, 


. ¥ : 
The mean and variance of * are given by 
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BY, = Hy, = Hy + Hay +7 Be, 


var (¥J= a, =£(>(X,-a) 


= Sai + SY 4) -4) 
im] dem] jel jd 
- Ath eth 


and . A, and ¥, are ndippalenrtin i= 


Thus we can determine the mean and the variance of y, 2, 


Can we guess about the probability distribution of Y, q 
The central limit theorem (CLT) provides an answer to this question. 


a) 
The CLT states that under very general conditions ae in distribution to 


Y~ N( My, Sy) as >. The conditions are: 
4 Mar Xn are independent and identically distributed. 


Ky; Kye, Z, : : ; 
bae"® are independent with same mean and variance, 


1. The random variables 


2. The random variables 
but not identically distributed. 


3. The random variables Ay Mayon hy are independent with different means and 
same variance and not identically distributed. 


4. The random variables a i ea are independent with different means and 


each variance being neither too small nor too large. 


We shall consider the first condition only. In this case, the central-limit theorem can be stated as 
follows: 


Proof of the Central Limit Theorem: 


We give a less rigorous proof of the theorem with the help of the characteristic function. 
Further we consider each of “1: *2:»¥n to have zero mean. Thus, fn = (4) +42 +... + 4,)/ vn. 
Hy, x Q, 
of = oh 


3y = 3 
Clearly, HU, ) = aut Jiafe and so on, 


The characteristic function of Y, is given by 


104 


We will show that as * —© the characteristic function Oy is of the form of the characteristic 
function of a Gaussian random variable. 


: Joly, : 
Expanding ® ~~“ in power series 


gi = 1 + JOS, + 


: a : 3 
2: Ae + a %3 # a, 


Assume all the moments of h to be finite. Then 
= fekyy _ : (jay? 2 (ja) : 3 
hla) Ble 14 Joy, + HO RORY + A BOD 


My =O and #(KA)=a7 =ag, we get 


fry 


Substituting 
dy (co) =1-(@ 2k + R(@,n) 


R(@,”) 


3 
where is the average of terms involving ® and higher powers of @ . 


R(@,) 


Note also that each term in involves a ratio of a higher moment and a power of ” and 


therefore, 
lim Ria) = 0 


o _ tat 
J tim by, (@)=1-— ox se 3 


which is the characteristic function of a Gaussian random variable with 0 mean and variance 
oy 
a 
¥, —— > C0, Te ; 


OPERATIONS ON MULTIPLE RANDOM VARIABLES 


Expected Values of Functions of Random Variables 


If Y= g(X) is a function of a continuous random variable 4, then 
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¥=g(X) EY-= Bg(X) = >, g(x) py (a) 
If g is a function of a discrete random variable ~~’ then vey 


Suppose Z= @(X.¥) is a function of continuous random variables © and then the 
expected value of Z is given by 


EZ = Eg(X,y) = f ofy(z)dz 


0. 


=] fe@rfar(e.y)dady 


Thus £2 can be computed without explicitly determining Fz @) , 


We can establish the above result as follows. 


Suppose Z=@(X.¥) has '? roots (4.94) 4 me a Z=Z.Then 


{z<Zsz+dz}=|J{(x,y)€ AD} 
i=l 
Where 


an Is the differential region containing (4.94): The mapping is illustrated in Figure 1 


for? = 3, 


{z <ZS2z+Az} 


Figure 1 
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Note that 


P(z<Zizthe))=f,(2d2= DY fer(a, Andy, 
(~HWeD 


i afte Le = a hy y(%,¥; Am AY, 


(%,.% jeaD 


™ > BG) F ey YAR AY, 


(XH ead 


As Z is varied over the entire 4 axis, the corresponding (non-overlapping) differential regions 
in + ~ ¥ plane cover the entire plane. 


“. fafa@)de = ff a @r)fex(sy)dedy 


0 


Thus, 
#£e(Xy)= J J ey) Sey (x yv)dady 
If Z= g(4,P) is a function of discrete random variables ¥ and Y , we can similarly show that 


BZ=Eg(XY)= 2D p EP xr Hy) 


x PeR yx 


Example 1 The joint pdf of two random variables X and Y is given by 


fa y(t) = 9 O<x<2, 0< <2 


=0 otherwise 
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= 2 
Find the joint expectation of BAP) AP 


Be(X,¥) = EX°*Y 


= fala rSyy (x. axdy 


ry | 
= [fx y—aydxdy 
00 4 


12 2 
= —{xdxfy'dy 
49 a 


Example 2 If Z=aX+bY, wherea and are constants, then 


BZ =aBX +bEY 


Proof: 
BZ = [ J (ax dy) fyy(aydady 


=f J axfyyG.yidady+ f J dyfyy (a, y)dxdy 
= [ax] fyy(x,yidvdxt [dy fyy (x, vdady 


=a J afy(xjdx tb | yf, Ody 
= aHX + bRY 
Thus, expectation is a linear operator. 


Example 3 


Consider the discrete random variables ¥ 894 ¥ discussed in Example 4 in lecture 18.The 
joint probability mass function of the random variables are tabulated in Table . Find the joint 


expectation of & (X.Y) = XY 
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ie a al 
Po [oes ot | ots Tos 
py(x) | 0399 045 


Clearly, AX¥= ¥ 2 YEO, VpPry&, ¥) 


ay eRe 
= 1|x1x0.3541«20.01 
=0.37 


Remark 
(1) We have earlier shown that expectation is a linear operator. We can generally write 


Baye, (4,7) + ayg.(4, 7) =a, 2g (4,2) + a,8g,(4, 1) 


ahs E(XY +5log, XY) = EXY +5Hlog, XY 


(2) If X and ¥ are independent random variables and B®, ¥) = aa. ,then 


£g(X,¥) = £g (X)g,%) 


=f fae, fry(x ax 


-wo-o 


=f fg Qe, fr@faxdy 


-o- 


La@)fr@dr | aM ow 
= £g,( 4) £g, (Y) 


Joint Moments of Random Variables 


Just like the moments of a random variable provide a summary description of the random 
variable, so also the joint moments provide summary description of two random variables. For 


two continuous random variables * and ¥ , the joint moment of order ” * # is defined as 


BX") = ff PY fey(aydxay 


And the joint central moment of order +” is defined as 
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B(X— pygl" @- ny) = ff e—ey)" yay fey (m dedly 


where #x = 2% ang Hy = BY 


Remark 


(1) If ¥ and ¥ are discrete random variables, the joint expectation of order ® and #7 is 
defined as 


EQX™Y*)= FE Ex*y "py sy) 
(7 ERY 5 
E(X — uy)" - ey)" = uote DX (x- By)" (y- By) Py (%) 


(2) If # =land #=1, we have the second-order moment of the random variables 
X and ¥ given by 


i 1 fy y (x, Yidady if ¥ and ¥ are continuous 
E(XV) = 0 0 


= LWPxryl(s%y) if ¥Y and ¥ are discrete 
(x ple Ry ¢ ; 


(3) If © and Y are independent, E(XY) = EXEY 


Covariance of two random variables 


The covariance of two random variables * and ¥ is defined as 


Cov.X,Y) = EX - uy X¥- py) 


Cov(X, Y) is also denoted as Oxe 
Expanding the right-hand side, we get 
Cov(X,¥) = BUX - py)(F - py) 


= BAY = py - yl + yy) 
= BAY ~ jy BX - Hy BY + wy iy 


The ratio is called the correlation coefficient. 
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Ip Pxr - then ¥ and ¥ are called positively correlated. 


Ip Pee a then ¥ and ¥ are called negatively correlated 


Ip Oxy ~ 0 then XY and ¥ are uncorrelated. 


We will also show that le om, YyIs _ To establish the relation, we prove the following result: 


2 Z 2 
For two random variables 4 and ¥ E(AY) Ss EX EY 


Proof: 
Consider the random variable 2 = 44 + ¥ 


B(aX +¥)? 20 
=> a? EX? + EY? +2aEXY 20_ 


Non-negativity of the left-hand side implies that its minimum also must be nonnegative. 


For the minimum value, 
dEZ* _ 
ada 


so the corresponding minimum is 


=> 


EB’ yy BE’ xY 
+ #Y' -2 


2 


ard ta 


BE’ XY 
Ex’ 


Since the minimum is nonnegative, 


BE’ XY 
Pal) 


= E°XY < BX'EyY* 
= |err|< Jax? fay’ 


Now 
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aX1) = LED 


oyoy 
— BAe By ~ By) 
af BCE ~ sxc)! ECF - pay)’ 
joer 1p) = BA 40 


BCE ~ oy) BOP - py 


: f BCE - ity)! fBCY- 9" 
JECT - wy) ECP - py) 
=1 
thus POS 


Uncorrelated random variables 


Two random variables * and ¥ are called uncorrelated if 


Cov’, F) =0 
which also means 


BUAY ) Sig Hy 


Recall that if ¥ and ¥ are independent random variables, then Fax @ Y)= Fx fr). 


EXY = i (i why y la Vodady assuming Y and F are continuous 


- ff wth aed 


= f fpr fe Oey 
then = EXEY 


Thus two independent random variables are always uncorrelated. 


Note that independence implies uncorrelated. But uncorrelated generally does not imply 
independence (except for jointly Gaussian random variables). 


Joint Characteristic Functions of Two Random Variables 


The joint characteristic function of two random variables X and Y is defined by 


Py y (a, @, } = Egle iow 


If * and ¥ are jointly continuous random variables, then 
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mo 


by y (@,, By) = f [ fax oe ade 


Note that Py, @,) is same as the two-dimensional Fourier transform with the basis function 
a aah + Jory} 
instead of 
a —l fea H+ Joa yF | . 
i xy (X.¥). 


is related to the joint characteristic function by the Fourier inversion formula 


om 


fala = Zz f [ter(@.a)e dada, 


If * and ¥ are discrete random variables, we can define the joint characteristic function in terms 
of the joint probability mass function as follows: 


fex(a)= F Epzyle.net? 


(XP eRe By 


Properties of the Joint Characteristic Function 


The joint characteristic function has properties similar to the properties of the chacteristic 
function of a single random variable. We can easily establish the following properties: 


1. Py (@) = Py y(@, 0) 
> f(@) = bey 0.0) 


3. If * and F are independent random variables, then 


jotta 


by y( a, @,) — He 
= B(ett piety 
a Feit® Aojek 


= By (By (ay) 


4. We have, 
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lak ty 
by (Qh, @,) = git tp, 


it ‘ 3 
= E+ ja.X + jay +2 (QA + jal) + 
2 
td mrt 
=1+ja8X+ jo,gy+2 = = 

Hence, 


bey 0,0) =1 


1a 
HX = —— Oy (a, @) 
Jaa, 


ea al 


re) 
BY = ——— Gy (ah, Gy) 
Jj Oa, oe 
BEY = a: A" by yh, a) 


“a 


f Ia Ia, ae, 


(792 + 22) 


In general, the order joint moment is given by 


1 BO" dy y (ah, By} 


EXMY™ = s+ ” ti 
f dard ay 


aye 


Example 2 The joint characteristic function of the jointly Gaussian random variables “ and 
¥ with the joint pdf 


“aa e) EE] 
all-pay rl] ry x tr or 


Jy yl ¥) = 


Let us recall the characteristic function of a Gaussian random variable 


X ~ NL. OR) 
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x es by 


cbt 


= piicie aya" 2 x] 


= piteieaiiat a 
If * and ¥ is jointly Gaussian, 


? 
Rare: | ig ad 
atl pyr] ey "| ey Fr wr 


e 
Fey (@y) = 
LTT y Ty fl ~ Pry 


we can similarly show that 


dy yl Qh, By) = Bellta tery 


. | 
- ite Ha apy TON py por yw penyam, HT | 


We can use the joint characteristic functions to simplify the probabilistic analysis as illustrated 
on next page: 


Jointly Gaussian Random Variables 


Many practically occurring random variables are modeled as jointly Gaussian random variables. 
For example, noise samples at different instants in the communication system are modeled as 
Jointly Gaussian random variables. 


Two random variables 4 and ¥ are called jointly Gaussian if their joint probability density 


1 [itag? 6 Uy Nay) a 
B a, a pier eee 
1 al A yl a ao a 
Jax\0.))* Tara Pay rie e , Wer <O0, O< y coo 
ake Ur aw 
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The joint pdf is determined by 5 parameters 


¢ means “x Hy 


oe and ot 
e variances ~~ Pv 


¢ correlation coefficient “*-” 


We denote the jointly Gaussian random variables * and ¥ with these parameters as 
(X,Y) ~ Mux, My, Ox Oy, Oxy) 


The joint pdf has a bell shape centered at Cx Hy) as shown in the Figure 1 below. The 


2 2 
variances °X ™*°Y determine the spread of the pdf surface and ©X,¥ determines the orientation 


of the surface in the * ~ ¥ plane. 


Figure 1 Jointly Gaussian PDF surface 


Properties of jointly Gaussian random variables 


(1) If “ and F are jointly Gaussian, then * and ¥ are both Gaussian. 
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We have 


Fe™ J fey BAY 


ceatil CaP a ny Mae KIM) (Py P 
e an ey gy 


(pay Pate) (Pay F 
ay —t — 


is © ( Xpay } 
oe aa x (- LEZ BOE 
= oe 1 F ara oF ra [ dy 


Similarly 
os (y) - Tam é 


(2) The converse of the above result is not true. If each of * and ¥ is Gaussian, and ¥ are 
not necessarily jointly Gaussian. Suppose 


ie 2 La 
1 Fall ey. OF 


Fxy(%¥) = tee F +sin xsin y) 


Fx yy) : 


in this example is non-Gaussian and qualifies to be a joint pdf. Because, 


> 
Jxy(%y) 2 end 
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a1 [tray tet 2 
iE my or 


la + sin xsin yidvalx 


a tale e an a 


wy *r avalx + 1 Ng a : : aye 
aba [ [re Sin Sit Va ver 
oD 


Tamu. io Tea3* 


wo + . 
=1+;1— fe eit edhe fe * siny dy 


in 


——————,-——_ ~~ 
ihegrition & an odd frectia 


The marginal density Fx) is given by 


1 |= 5 =") 


=} l ep oF ha. 2 
yaa} for? (1+ sin xsin yidy 


teal rena o fara} ret 
OE te ee 
rs Ly it EN sin xsin wey 


=o 


ittegrahor of arodd fine tion 


3) or. 
Similarly, Fy) aay 


Thus “ and ¥ are both Gaussian, but not jointly Gaussian. 


(3) If * and ¥ are jointly Gaussian, then for any constants @ and ® the random variable 


, : ; ' = + : 
Z given by 2 = @X + 2Y is Gaussian with mean “2 © @#x bby and variance 
Oy) ard, + ba? + 2abt yy Oy y 


(4) Two jointly Gaussian RVs * and ‘are independent if and only if * and Fare 


uncorrelated (exy = 9) Observe that if © and Yare uncorrelated, then 
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-} [Ropes Fay " 
x *Y 
Sex Gd) raga, e 


(py P (Pm tty 1? 
= 1 


ae 
V2moy 


ex Fy 


= fee O) 


Example 1 Suppose X and Y are two jointly-Gaussian 0-mean random variables with variances 


of 1 and 4 respectively and a covariance of 1. Find the joint PDF Jz. (%.) 


Hy= iy = 0, oy =1, oy = 4and cov(X,Y) = 1. 
Cov(X,¥) 1 _1 
Ty, 1X2 2 


ssa a 


We have 


Example 2 Linear transformation of two random variables 
Suppose 2 = @¥ + 5Y then 


t,(@) = Eel? = elle = 9 | (a@,ba) 


If * and ¥ are jointly Gaussian, then 


by (@) = & y(a@, 2a) 
J dig te-Hatoh +2 py abe 6, +d4o? ja? 
S32 


Which is the characteristic function of a Gaussian random variable 


. 3 2 2 2 
with mean #z - 4x +4. and variance [ = Fy +2@y yo yoy + Oy 


thus the linear transformation of two Gaussian random variables is a Gaussian random 
variable. 
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Example 3 If Z = X + Y and X and Y are independent, then 
$y (@) = by y(@, @) 


= by (@) G (@) 


Using the property of the Fourier transform, we get 


Fz) = fx(2)* fr) 


Hence proved. 


Univariate transformations 
When working on the probability density function (pdf) of a random variable X, one is 


often led to create a new variable Y defined as a function f(X) of the original variable X. For 
example, if X~N(u, @7), then the new variable: 


Y=f(X) =(X- w/e 
Is N (0, 1). 


It is also often the case that the quantity of interest is a function of another (random) 
quantity whose distribution is known. Here are a few examples: 
*Scaling: from degrees to radians, miles to kilometers, light-years to parsecs, degrees 


Celsius to degrees Fahrenheit, linear to logarithmic scale, % to the distribution of the variance 
* Laws of physics: what is the distribution of the kinetic energy of the molecules of a gas if 
the distribution of the speed of the molecules is known ? 


So the general question is: 
* Tf Y=h(X), 
* And if f(x) is the pdf of X, 
Then what is the pdf g(y) of Y? 
TRANSFORMATION OF A MULTIPLE RANDOM VARIABLES 


Multivariate transformations 


The problem extends naturally to the case when several variables Y; are defined from 
several variables X; through a transformation y = h(x). 
Here are some examples: 
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Rotation of the reference frame 


Let f(x, y) be the probability density function of the pair of r.v. {X, Y}. Let's rotate 
the reference frame {x, y} by an angle@. The new axes {x’, y’} define two new r. v. {X’, 
Y'}. What is the joint probability density function of {X’, Y'}? 


Polar coordinates 


Let f(x, y) be the joint probability density function of the pair of r. v. {X, Y}, 
expressed in the Cartesian reference frame {x, y}. Any point (x, y) in the plane can also be 
identified by its polar coordinates (r,@). So any realization of the pair {X, Y} produces a 
pair of values of r and@, therefore defining two new r. v. R and@. 

What is the joint probability density function of R and? What are the (marginal) 
distributions of R and of? 


Sampling distributions 


Let f(x) is the pdf of the r. v. X. Let also Z; = z1(x1, x2... X,) be a Statistic, e.g. the 
sample mean. What is the pdf of Z)? 
Z, is a function of the n r. v. X; (with n the sample size), that are lid with pdf f(x). If it is 
possible to identify n - 1 other independent statistics Zi, i = 2... n, then a transformation Z = 
h(X) is defined, and g(z), the joint distribution of Z = {Z1, Zo, ..., Z,} can be calculated. 
The pdf of Z; is then calculated as one of the marginal distributions of Z by integrating g(z) 
over zj, i= 2, .., Nn. 


Integration limits 


Calculations on joint distributions often involve multiple integrals whose 
integration limits are themselves variables. An appropriate change of variables sometimes 
allows changing all these variables but one into fixed integration limits, thus making the 
calculation of the integrals much simpler. 


Linear Transformations of Random Variables 


A linear transformation is a change to a variable characterized by one or more of the 
following operations: adding a constant to the variable, subtracting a constant from the variable, 
multiplying the variable by a constant, and/or dividing the variable by a constant. 


When a linear transformation is applied to a random variable, a new random variable is 
created. To illustrate, let X be a random variable, and let m and b be constants. Each of the 
following examples show how a linear transformation of X defines a new random variable Y. 


Adding a constant: Y = X +b 


Subtracting a constant: Y = X - b 
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Multiplying by a constant: Y = mX 
Dividing by a constant: Y = X/m 
Multiplying by a constant and adding a constant: Y = mX + b 


Dividing by a constant and subtracting a constant: Y = X/m - b 


= T 
Suppose the vector of random variables X= (X%).----XnY has the joint 
distribution’G) = f01-----97), get ¥ = AX+ Brox some square matrix 4 and vector®. If 
1 f4ley— 
deté + Othen Yhas the joint distribution sual (A Y B)). 


Indeed, suppose ¥-2v) (this is the notation for "the 2) is the distribution density of ¥ ") and 
XFO)_ For any domain D of the ¥— space we can 


|, 2@)¢y = Prob(Y € D) = Prob(4X+ B < D) = 


write 
= Prob(X < 4"(D—B)) = Wesco PO = 


y=Ax+B 


We make the change of variables 


in the last integral. 


= fe f(A“"(y—B)) Bo lay - {, fA Ge B)) 51 wy. (Linear transformation of 


x) 
Dy) 


random variables) 


2 
The linear transformation 94 + His distributed as (ue ° ), The & was defined in the section ( 
Definition of normal variable). 


For two independent standard normal variables (s.n.v.) é1 and €2 the combination 7 161 +0262 


N(0. i? +03 | 


A product of normal variables is not a normal variable. See the section on the chi - 
squared distribution. 


is distributed as 
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UNIT IV 
Stochastic Processes-Temporal Characteristics 


The Stochastic process Concept 

Classification of Processes, Deterministic and Nondeterministic Processes 
Distribution and Density Functions 

Statistical Independence 

Concept of Stationarity: First-Order Stationary Processes, Second-Order and Wide-Sense 
Stationarity, Nth-Order and Strict-Sense Stationarity 

Time Averages and Ergodicity 

Mean-Ergodic Processes 

Correlation-Ergodic Processes 

Autocorrelation Function and Its Properties 

Cross-Correlation Function and Its Properties 

Covariance Functions and its properties 

Gaussian Random Processes. 


Linear system Response: 


Mean 

Mean-squared value 
Autocorrelation 
Cross-Correlation Functions 


RANDOM PROCESSES: TEMPORAL CHARACTERISTICS 


The random processes are also called as stochastic processes which deal with 
randomly varying time wave forms such as any message signals and noise. They are 
described statistically since the complete knowledge about their origin is not known. So 
Statistical measures are used. Probability distribution and probability density functions give 
the complete statistical characteristics of random signals. A random process is a function of 
both sample space and time variables. And can be represented as {X<x(s,t)}. 


Deterministic_and Non-deterministic processes: In general a random process may be 
deterministic or non deterministic. A process is called as deterministic random process if 


future values of any sample function can be predicted from its past values. For example, X(t) 
= A sin (@ot+®), where the parameters A, @) and © may be random variables, is deterministic 
random process because the future values of the sample function can be detected from its 
known shape. If future values of a sample function cannot be detected from observed past 
values, the process is called non-deterministic process. 


Classification of random process: Random processes are mainly classified into four types 
based on the time and random variable X as follows. 


1. Continuous Random Process: A random process is said to be continuous if both the 
random variable X and time t are continuous. The below figure shows a continuous 
random process. The fluctuations of noise voltage in any network is a continuous 
random process. 


\ 
pH eek! A eee ee 
| Rui Met 2s olde rbaye ee 


\_SJ 


(a) | Sampie function of a continuous random process 


2. Discrete Random Process: In discrete random process, the random variable X has 
only discrete values while time, t is continuous. The below figure shows a discrete 
random process. A digital encoded signal has only two discrete values a positive level 
and a negative level but time is continuous. So it is a discrete random process. 


YE SP! hs irate pa —+ 


Sample function of a discrete random process 


3. Continuous Random Sequence: A random process for which the random variable X is 
continuous but t has discrete values is called continuous random sequence. A 
continuous random signal is defined only at discrete (sample) time intervals. It is also 
called as a discrete time random process and can be represented as a set of random 
variables {X(t)} for samples ty, k=0,+1,+2..... 


4. Discrete Random Sequence: In discrete random sequence both random variable X and 
time t are discrete. It can be obtained by sampling and quantizing a random signal. 
This is called the random process and is mostly used in digital signal processing 
applications. The amplitude of the sequence can be quantized into two levels or multi 
levels as shown in below figure s (d) and (e). 


+ 1 ite 


(d) | Sample function of a discrete random sequence 


epee CHIT, 


(e) Sample function of adiscrete random sequence (multi levels) 


Joint distribution functions of random process: Consider a random process X(t). For a 
single random variable at time t;, X;=X(t,), The cumulative distribution function is defined as 
Fx(xy;ty) = P {(X(t,)S<x1} where x, is any real number. The function F'x(x;;t;) is known as the 
first order distribution function of X(t). For two random variables at time instants t; and tz 
X(t,) = X, and X(t) = Xo, the joint distribution is called the second order joint distribution 
function of the random process X(t) and is given by 


Fx(X1, Xo; ty, te) = P {(X(ti)Sx1, X(to)<x2}. In general for N random variables at N time 
intervals X(t,) = X; i=1,2,...N, the N" order joint distribution function of X(t) is defined as 


Xn} ty, to,..... ty) = P {((X(t)Sx1, X(t2)Sx2,.... X(ty)Sxn}. 


Joint density functions of random process:: Joint density functions of a random process 
can be obtained from the derivatives of the distribution functions. 


dF x (x1;t1) 


1. First order density function: fx(x;;t;) = = 
1 


07 Fx(x1,x2 ; t1,t2) 


2. Second order density function: fx(x1, x2; ti, tz) = Ais 
1 2 


3. N" order density function: fx(X1, X2...... Xn} ty, to,..... tn) = 


Independent random processes: Consider a random process X(t). Let X(ti) = Xi, i= 1,2,...N 
be N Random variables defined at time constants t),t2, ... t 1 with density functions fx(x,;t,), 
fx(Xp;t2), ... fx(xn ; tn). If the random process X(t) is statistically independent, then the Ni 
order joint density function is equal to the product of individual joint functions of X(t) ie. 


fx(X1, X20. XN 5 t, t...... tn) = fx(x1;t1) fx(X25t2). ees fx(Xn 3 ty). Similarly the joint distribution 
will be the product of the individual distribution functions. 


Statistical properties of Random Processes: The following are the statistical properties of 
random processes. 


1. Mean: The mean value of a random process X(t) is equal to the expected value of the 
random process i.e. X(t) = E[X(t)] = ae xf. (x; thdx 

2. Autocorrelation: Consider random process X(t). Let X; and Xj be two random 
variables defined at times t, and t2 respectively with joint density function 
fx(X1, X2; ti, to). The correlation of X; and X2, E[X; X2] = E[X(ti) X(t2)] is called the 
autocorrelation function of the random process X(t) defined as 
Rxx(ti,t2) = E[X1 X2] = E[X(t) X(t2)] or 


Rxx(ttz) = fo So, XiX2fk (XL, x2 5 t1, t2) dx,dx, 


3. Cross correlation: Consider two random processes X(t) and Y(t) defined with 
random variables X and Y at time instants t; and ty respectively. The joint density 
function is f,y(x,y ; t1,t2).Then the correlation of X and Y, E[XY] = E[X(t;) Y(t2)] is 
called the cross correlation function of the random processes X(t) and Y(t) which is 
defined as 
Rxy(tz,tz) = E[X Y] = ELX(t,) Y(t2)] or 


Rxv(ttz) = fo xy fey (%y 5 t1, t2) dxdy 


Stationary Processes: A random process is said to be stationary if all its statistical properties 
such as mean, moments, variances etc... do not change with time. The stationarity which 
depends on the density functions has different levels or orders. 


1. First order stationary process: A random process is said to be stationary to order 
one or first order stationary if its first order density function does not change with 
time or shift in time value. If X(t) is a first order stationary process then 


fx(xyt,) = fx(x1jt,+At) for any time t;. Where At is shift in time value. Therefore the 
condition for a process to be a first order stationary random process is that its mean 
value must be constant at any time instant. i.e. ELX(t)] = X = constant. 


2. Second order stationary process: A random process is said to be stationary to order 
two or second order stationary if its second order joint density function does not 
change with time or shift in time value i.e. fx(X1, X25 ti, to) = fx(X1, Xo;ti+At, to+At) for 
all t1,t2 and At. It is a function of time difference (tz, t1) and not absolute time t. Note 
that a second order stationary process is also a first order stationary process. The 
condition for a process to be a second order stationary is that its autocorrelation 
should depend only on time differences and not on absolute time. i.e. If 
Rxx(ti,tz2) = E[X(t,) X(t2)] is autocorrelation function and t =t, -t; then 
Rxx(ty,ti+ 1) = ELX(t,) X(t+ 1)] = Rxx(t) . Rxx(t) should be independent of time t. 

3. Wide sense stationary (WSS) process: If a random process X(t) is a second order 
stationary process, then it is called a wide sense stationary (WSS) or a weak sense 
stationary process. However the converse is not true. The condition for a wide sense 
stationary process are 1. E[X(t)] = X = constant. 2. E[X(t) X(t+t)] = Rxx(t) is 
independent of absolute time t. 

Joint wide sense stationary process: Consider two random processes X(t) and Y(t). If 
they are jointly WSS, then the cross correlation function of X(t) and Y(t) is a function 
of time difference t =tz -tjonly and not absolute time. i.e. Rxy(ti,t2) = E[X(t1) Y(t2)] . 
If t =tz -t; then Rxy(t,t+ t) = E[X(t) Y(t+ t)] = Rxy(t). Therefore the conditions for a 
process to be joint wide sense stationary are 1. E[ X(t)] = X = constant. 

2. E[Y(t)] = Y = constant 3. E[X(t) Y(t+ t)] =Rxy(t) is independent of time t. 


4. Strict sense stationary (SSS) processes: A random process X(t) is said to be strict 


Sense stationary if its Nth order joint density function does not change with time or 
shift in time value. i.e. 

fx(X1, XxX... Xn; ty, to... tn) = fx(X1, X2. Xn; ty tAt, to+At,... tnt At) for all ti, to... tw 
and At. A process that is stationary to all orders n=1,2,. . . N is called strict sense 
stationary process. Note that SSS process is also a WSS process. But the reverse is not 
true. 


Time Average Function: Consider a random process X(t). Let x(t) be a sample function 
which exists for all time at a fixed value in the given sample space S. The average value of 
x(t) taken over all times is called the time average of x(t). It is also called mean value of x(t). 


It can be expressed as x = A[x(t)] = lim... = fe x(t)dt. 

Time autocorrelation function: Consider a random process X(t). The time average of the 
product X(t) and X(t+ 1) is called time average autocorrelation function of x(t) and is denoted 
as Rax(t) = A[X(t) X(t+1)] oF Rex(t) = limp so fi, x(Ox(t + Dat. 

Time mean square function: If t = 0, the time average of x°(t) is called time mean square 
value of x(t) defined as = A[X°(t)] = limy,.0 = ie x? (t)dt. 

Time cross correlation function: Let X(t) and Y(t) be two random processes with sample 


functions x(t) and y(t) respectively. The time average of the product of x(t) y(t+ Tt) is called 
time cross correlation function of x(t) and y(t). Denoted as 


Ryy(t) = limp soo J, XO y(t + Dat. 


Ergodic Theorem and Ergodic Process: The Ergodic theorem states that for any random 
process X(t), all time averages of sample functions of x(t) are equal to the corresponding 
statistical or ensemble averages of X(t). i.e. X =X or Rxx(t) = Rxx(t) . Random processes that 
satisfy the Ergodic theorem are called Ergodic processes. 


Joint Ergodic Process: Let X(t) and Y(t) be two random processes with sample functions 
x(t) and y(t) respectively. The two random processes are said to be jointly Ergodic if they are 
individually Ergodic and their time cross correlation functions are equal to their respective 
statistical cross correlation functions. ie. 1. ¥ = X, V = Y 2. Rux(t) = Rxx(t), Ray(t) = Rxy(t) 
and Ryy(t) = Ryy(t). 


Mean Ergodic Random Process: A random process X(t) is said to be mean Ergodic if time 
average of any sample function x(t) is equal to its statistical average, X which is constant and 
the probability of all other sample functions is equal to one. i.e. E[X(t)] = X = A[x(t)] = x 
with probability one for all x(t). 

Autocorrelation Ergodic Process: A stationary random process X(t) is said to be 
Autocorrelation Ergodic if and only if the time autocorrelation function of any sample 
function x(t) is equal to the statistical autocorrelation function of X(t). i.e. A[x(t) x(t+t1)] = 


E[X(t) X(t+t)] or Rxx(t) = Rxx(1). 


Cross Correlation Ergodic Process: Two stationary random processes X(t) and Y(t) are 
said to be cross correlation Ergodic if and only if its time cross correlation function of sample 
functions x(t) and y(t) is equal to the statistical cross correlation function of X(t) and Y(t). ie. 


ALx(t) y(t+t)] = ELX(t) Y(t+2)] or Ryy(t) = Rxy(t). 


Properties of Autocorrelation function: Consider that a random process X(t) is at least 
WSS and is a function of time difference t = t2-t). Then the following are the properties of the 
autocorrelation function of X(t). 


1. Mean square value of X(t) is E[X7(t)] = Rxx(0). It is equal to the power (average) of 
the process, X(t). 
Proof: We know that for X(t), Rxx(t) = E[X(t) X(t+ 1)] . If t = 0, then Rxx(0) = E[X(t) 
X(t)] = E[X7(t)] hence proved. 
2. Autocorrelation function is maximum at the origin i.e. |[Rxx(t)| < Rxx(0). 
Proof: Consider two random variables X(t) and X(t) of X(t) defined at time intervals 
t, and tp respectively. Consider a positive quantity [X(t,) +X(to)]° >0 
Taking Expectation on both sides, we get E[X(t:) +X(t2)]? > 0 
E[X°(ty)+ X*(to) + 2X(ty) X(t)] = 0 
E[X*(t)}+ E[X°() + 2E[X(ty) X(t)] = 0 
Rxx(0)+ Rxx(0)t 2 Rxx(ti,tz) > 0 [Since E[X*(t)] = Rxx(0)] 
Given X(t) is WSS and T = to-ty. 
Therefore 2 Rxx(O+ 2 Rxx(t) = 0 
Rxx(0+ Rxx(t) > Oor 
[Rxx(t)| < Rxx(0) hence proved. 
3. Rxx(t) is an even function of Tt i.e. Rxx(-T) = Rxx(T). 
Proof: We know that Rxx(t) = E[X(t) X(t+ T)] 
Let t= - t then 
Rxx(-t) = E[X(t) X(t- 1)] 
Let u=t- t or t= ut T 
Therefore Rxx(-t) = E[X(u+ t) X(u)] = E[X(u) X(u+ 1)] 


Rxx(-t) = Rxx(t) hence proved. 
4. If arandom process X(t) has a non zero mean value, E[X(t)] # 0 and Ergodic with no 
periodic components, then limjz|o0 Rxx(T) = Ke, 
Proof : Consider a random variable X(t)with random variables X(t,) and X(t2). Given 
the mean value is E[X(t)] = X #0. We know that 
Rxx(t) = E[X(t)X(t+t)] = E[X(ti1) X(t2)]. Since the process has no _ periodic 
components, as|t| > 00, the random variable becomes independent, i.e. 
lim|z}00 Ryx(t) = ELX(ti) X(t2)] = ELX(t)] EL X(t2)] 
Since X(t) is Ergodic E[X(t;)] = E[ X(tz)] =X 
Therefore lim)z)0 Rxx(T) = X? hence proved. 
5. If X(t) is periodic then its autocorrelation function is also periodic. 
Proof: Consider a Random process X(t) which is periodic with period To 
Then X(t) = X(t+ To) or 
X(t+ T) = X(t+t + To). Now we have Rxx(t) = E[X(t)X(t+1)] then 
Rxx(t+ To) = ELX(t)X(t+t+ To)] 
Given X(t) is WSS, Rxx(t+ To) = E[X(t)X(t+1)] 
Rxx(tt To) = Rxx(t) 
Therefore Rxx(t) is periodic hence proved. 
6. If X(t) is Ergodic has zero mean, and no periodic components then 
lim)z| 5.00 Ryx(T) =0. 
Proof: It is already proved that lim;).. Rxx(t) = X*. Where X is the mean value of 
X(t) which is given as zero. 
Therefore lim); Rxx(t) = 0 hence proved. 
7. The autocorrelation function of random process Rxx(t) cannot have any arbitrary 
shape. 
Proof: The autocorrelation function Rxx(t) is an even function of t and has maximum 
value at the origin. Hence the autocorrelation function cannot have an arbitrary shape 
hence proved. 
8. If arandom process X(t) with zero mean has the DC component A as Y(t) =A + X(t), 
Then Ryy(t) = A*+ Rxx(t). 
Proof: Given a random process Y(t) =A + X(t). 
We know that Ryy(t) = E[Y(t)Y(t+t)] =E[(A + X(t)) (A + X(t+ 1))] 
- E[(A* + AX(t) + AX(tt+ 1+ X(t) X(t+ 1] 
- E[(A’] + AE[X(t)] + E[AX(t+ 1)]+ E[X(t) X(t+ 1)] 
=A*+0+0+ Rxx(t). 
Therefore Ryy(t) = A?+ Rxx(t) hence proved. 
9. If arandom process Z(t) is sum of two random processes X(t) and Y(t) 
i.e, Z(t) = X(t) at Y(t). Then Rz7(t) = Rxx(t)+ Rxy(t)+ Ryx(t)+ Ryy(t) 
Proof: Given Z(t) = X(t) + Y(t). 
We know that Rzz(t) = E[Z(t)Z(t+1)] 
= EL(X(t)+ Y(t) (X(t+7)¥ (t+) ] 
= E[(X(t) X(t+t)+ X(t) Y(t+t) + Y(t) X(t+t) +Y(t) Y(t+t))] 
= E[(X(t) X(t+t)]+ E[X(t) Y(t+t)] +E[Y(t) X(t+t)] +E[Y(t) Y(t+t))] 
Therefore Rzz7(t) = Rxx(t)+ Rxy(t)+ Ryx(t)+ Ryy(t) hence proved. 
Properties of Cross Correlation Function: Consider two random processes X(t) and Y(t) 
are at least jointly WSS. And the cross correlation function is a function of the time 
difference t = t2-t;. Then the following are the properties of cross correlation function. 
1. Rxy(t) = Ryx(-t) is a Symmetrical property. 
Proof: We know that Rxy(t) = E[X(t) Y(t+ t)] also 
Ryx(t) = ELY(t) X(t+ 1)] 
Let t=-t then 


Ryx(-t) = E[Y(t) X(t- 1)] 

Let u=t- t or t= ut T. then 

Ryx(-t) = E[Y(u+ t) X(u)] = E[X(u) Y(u+ 1)] 

Therefore Ryx(-t) = Rxy(t) hence proved. 

. If Rxx(t) and Ryy(t) are the autocorrelation functions of X(t) and Y(t) respectively 
then the cross correlation satisfies the inequality: |Rxy(t)| < ./Rxx(0)Ryy(0). 

Proof: Consider two random processes X(t) and Y(t) with auto correlation functions 

Rxx(t) and Ryy(t). Also consider the inequality 


X(t) Y(t+T) 42 
E => 
xO = Try)! = 
E[ x(t) rea (0) 42 X(t)Y¥(t+t) 1 
VRxx(0) + TO) ~ VRxx(0)Ryy(0)) 


X(t) y? to) X(t)Y¥ (t+t) 


VRxx(0) sl + os Ryv@)! ET, ax OR yy)! = 


We know that E[X°(t)] = Rxx(0) and E[Y°(t)] = Ryy(0) and E[X(t) X(tt+ t)] = Rxy(1) 


E[ 


Rxx(0) i: Ryy(0) Rxy() 


Rxyx(0)  Ryy(0)~ — -VRxx(0)Ryy(0) 


Therefore >0 


2 Rxy(t) 


~ — J Rxx(0)Ryy(0) ~~ 


Rxy(t) 


~ ¥Rxx(O)Ryy() ~ 
Vv Rxx(0)Ryy(0). = [Rxy(t)| Or 
IRxy(t)| S /Rxx(0)Ryy(0) hence proved. 


Hence the absolute value of the cross correlation function is always less than or equal 
to the geometrical mean of the autocorrelation functions. 
If Rxx(t) and Ryy(t) are the autocorrelation functions of X(t) and Y(t) respectively 


then the cross correlation satisfies the inequality: |Ryy(t)| < - [Rxx(0)+ Ryy(0)]. 


Proof: We know that the geometric mean of any two positive numbers cannot exceed 
their arithmetic mean that is if Rxx(t) and Ryy(t) are two positive quantities then at 
t=0, 


J Rxx(0)Ryy(0) <5 [Rxx(0)+ Ryy(0)].We know that |Rxy(t)| S$ /Rxx(0)Ryy(0) 
Therefore |Rxy(t)| < - [Rxx(0)+ Ryy(0)]. Hence proved. 


. If two random processes X(t) and Y(t) are statistically independent and are at least 
WSS, then Rxy(t) =X Y. 

Proof: Let two random processes X(t) and Y(t) be jointly WSS, then we know that 

Rxy(t) =E[X(t) Y(t+ t)] Since X(t) and Y(t) are independent 


Rxy(t) =E[X(HJEL ¥(t+ 1)] 


Therefore Rxy(t) =X Y hence proved. 
5. If two random processes X(t) and Y(t) have zero mean and are jointly WSS, then 
lim)z) 5.00 Ryy(t) =0. 
Proof: We know that Rxy(t) =E[ X(t) Y(t+ 1)]. Taking the limits on both sides 
lim)z)500 Ryy(t) = lim) z}5.00 E[X(t) Y(t + T)|. 
As |t| — 00, the random processes X(t) and Y(t) can be considered as independent 
processes therefore 
limjel-sco Rxy(t) = ELX(QIEL ¥(t+ 1)] =X 7 
Given X = Y=0 
Therefore lim)z).0. Rxy(t) = 0. Similarly lim);),. Ryx(t) = 0. Hence proved. 
Covariance functions for random processes: 
Auto Covariance function: Consider two random processes X(t) and X(t+ t) at two time 
intervals t and t+ t. The auto covariance function can be expressed as 
Cxx(t, ttt) = E[(X(t)-ELX(t)]) (X(t) — E[X(t+1)])] or 


Cxx(t, t+t) = Rxx(t, t+t) - E[(X(t) E[X(t+1)] 
If X(t) is WSS, then Cxx(t) = Rxx(t) - XA, AtTtT= 0, Cxx(0) = Rxx(0) = x? =E[X?]- Xe = aX? 


Therefore at t = 0, the auto covariance function becomes the Variance of the random process. 


The autocorrelation coefficient of the random process, X(t) is defined as 
Cxx (t,t+T) 


Pxx(t, t+t) = qe if t =0, 
Cxx (t,t) 
Pxx(0) = 5 AI. Also loxx(tt +0] $1. 


Cross Covariance Function: If two random processes X(t) and Y(t) have random variables 
X(t) and Y(t+ t), then the cross covariance function can be defined as 
Cxy(t, ttt) = E[(X(t)-E[X()]) (CY (t+) — E[Y(t+1)])] or 


Cxy(t, t+t) = Rxy(t, t+t) - E[(X(t) E[Y(t+1)]. If X(t) and Y(t) are jointly WSS, then 
Cxy(t) = Rxy(t) -X Y. If X(t) and Y(t) are Uncorrelated then 

Cxy(t, t+t) =0. 

The cross correlation coefficient of random processes X(t) and Y(t) is defined as 


Cxy (t,t+T) 


Pxv(t 1) = enc GrrD 


Cxy(0) — Cxy(0) 


Pxx(0)= V Cxx(0)Cyy (0) oxoy 


if t =0, 


Gaussian Random Process: Consider a continuous random process X(t). Let N random 
variables X,;=X(t,),X.=X(t2), . . .,Xn =X(tn) be defined at time intervals ty, to,. . . ty 
respectively. If random variables are jointly Gaussian for any N=1,2,.... And at any time 
instants t),to,. . . ty. Then the random process X(t) is called Gaussian random process. The 
Gaussian density function is given as 


1 —— —__ 


fxQX, Xo... XN5 ty, ty... ty) = Gm N2\[Cxx] 12/2 exp(-[X — X]"[Cyx]*[X — X])/2 
where Cx x is a covariance matrix. 


Poisson’s random process: The Poisson process X(t) is a discrete random process which 
represents the number of times that some event has occurred as a function of time. If the 
number of occurrences of an event in any finite time interval is described by a Poisson 
distribution with the average rate of occurrence is A, then the probability of exactly 
occurrences over a time interval (0,t) is 


= e 


P[X(t)=K] = , K=0,1,2, . 


And the probability density function is 


Gike* 


fel) = Deo 5 Oh. 


UNIT V 
Stochastic Processes-Spectral Characteristics 


The Power Spectrum and its Properties 

Relationship between Power Spectrum and Autocorrelation Function 

The Cross-Power Density Spectrum and Properties 

Relationship between Cross-Power Spectrum and Cross-Correlation Function. 


Spectral characteristics of system response: 


e Power density spectrum of response 
e Cross power spectral density of input and output of a linear system 


RANDOM PROCESSES:SPECTRAL CHARACTERISTICS 


In this unit we will study the characteristics of random processes regarding 
correlation and covariance functions which are defined in time domain. This unit explores the 
important concept of characterizing random processes in the frequency domain. These 
characteristics are called spectral characteristics. All the concepts in this unit can be easily 
learnt from the theory of Fourier transforms. 


Consider a random process X (t). The amplitude of the random process, when it varies 
randomly with time, does not satisfy Dirichlet’s conditions. Therefore it is not possible to 
apply the Fourier transform directly on the random process for a frequency domain analysis. 
Thus the autocorrelation function of a WSS random process is used to study spectral 
characteristics such as power density spectrum or power spectral density (psd). 


Power Density Spectrum: The power spectrum of a WSS random process X (t) is defined as 
the Fourier transform of the autocorrelation function Rxx (t) of X (t). It can be expressed as 


Sxx(@) = 1 Ryy@ e-jet gz We can obtain the autocorrelation function from the power 


ioe) 


spectral density by taking the inverse Fourier transform i.e. 


ioe) 


1 
Rxx (T) = ake Syy(w) eJwt dw 


Therefore, the power density spectrum Sxx(@) and the autocorrelation function Rxx (t) are 
Fourier transform pairs. 


2 
E(|X_(w)| ] 


The power spectral density can also be defined as Sxx() = limr_,., - 


Where X+7(@) is a Fourier transform of X(t) in interval [-T,T] 


Average power of the random process: The average power Pxx of a WSS random process 
X(t) is defined as the time average of its second order moment or autocorrelation function at 
t =0. 


Mathematically, Pxx= A {E[X°(t)]} 
= 1 T 2 
Pxx= limp 4055 S_, E[X?(t)]dt 
Or Pxx =Rxx (t)|t = 0 
We know that from the power density spectrum, 
1 io) 
Rxx (t) = ran Syy) eJ®T qy 
At t=0 Pxx = Rxx (0) == J", Syx@) aw 


Therefore average power of X(t) is 


1 ioe) 
Pxx = = J Sex) dw 


Properties of power density spectrum: The properties of the power density spectrum 
Sxx(@) fora WSS random process X(t) are given as 


(1) Sxx(@)= 0 


2, . 
Proof: From the definition, the expected value of a non negative function EL] Xp. i is 


always non-negative. 
Therefore Sxx(@)= 0 hence proved. 


(2) The power spectral density at zero frequency is equal to the area under the curve of 


the autocorrelation Rxx (t) i.e. Sxx(0) =f" Ryx@ ar 


Proof: From the definition we know that Sxx(@) = if Ryy@ e-jot gz at W=0, 
Sxx(0) = fe Ryy@ gq, hence proved 
(3) The power density spectrum of a real process X(t) is an even function i.e. 
Sxx(-@)= Sxx(@) 
Proof: Consider a WSS real process X(t). then 
Sxx(@) = f_, Ryy@ e-jot gz also Sxx(-0) =f" Ryy@ ior ar 
Substitute t = -t then 
Sxx(-@) = fee RyyCa e-jer gr 
Since X(t) is real, from the properties of autocorrelation we know that, Rxx (-t) = Rxx (1) 
Therefore Sxx(-@) =f" Ryy@ giot gr 
Sxx(-@)= Sxx(@) hence proved. 


(4) Sxx(@) is always a real function 


2 
EX co] J 
2T 


Proof: We know that Sxx(@) = limy.,., 
Since the function |x p(w) |" is areal function, Sxx(@) is always a real function hence proved. 


(5) If Sxx(@) is a psd of the WSS random process X(t), then 


— dc Digaelea ag ee {E[X?(t)]} = Rxx (0) or The time average of the mean square value of a 


WSS random process equals the area under the curve of the power spectral density. 


Proof: We know that Rxx (t) = A {E[X (t + t) X(t)]} 


1.5 xx) qw att =0, 


Rxx (0) = A {ELX*(t)]} = = feos. yx(@) dg = Area under the curve of the power spectral 
density. Hence proved. 
(6) If X(t) is a WSS random process with psd Sxx(@), then the psd of the derivative of 
X(t) is equal to «” times the psd Sxx(w). That is SXX() = w* Sxx(@). 


Proof: We know that Sxx(@) = limy_,, ec9| and X7(@) = fe X(the J? dt 


= Xq(@) =A = pT x(eJotdt 
=i X(t)= e Jetdt 

= f',.X(t)(—jw) eJ°tdt 

=(—jw) f7.X(t) e/tdt 


a eee 
Therefore SX¥X(@) =limy,.. aro" 
2 


=limy_x Be 


wE[|(-10)X 0) | 


= limy., x 


«6 4 2 
SXX(w) =w? limy.,,, AC") 


Therefore S¥X(w) = w* Sxx(@) hence proved. 


Cross power density spectrum: Consider two real random processes X(t) and Y(t). which 
are jointly WSS random processes, then the cross power density spectrum is defined as the 
Fourier transform of the cross correlation function of X(t) and Y(t).and is expressed as 

(oe) ioe) . . 
Sxy(@) = f_Ryy@ e-iotg, and Syx(m) = f_ Ryy@,-jorg, by inverse Fourier 
transformation, we can obtain the cross correlation functions as 


1 co 
Rxy (T) = — "5 xy) eJ0T dy) and Ryx (t) = oe Syy(w) eJT da 
Therefore the cross psd and cross correlation functions are forms a Fourier transform pair. 


If X7(@) and Y+7(@) are Fourier transforms of X(t) and Y(t) respectively in interval [-T,T], 
Then the cross power density spectrum is defined as 


1 1 


E[ 
Sxy(@) = limr_,., 


[ ] 


E ¥ PX plo) 


x (w)Y (w) ] 
L r and Syx(@) = limy. 


2T 2T 


Average cross power: The average cross power Pxy of the WSS random processes X(t) and 
Y(t) is defined as the cross correlation function at t =0. That is 


Pxy=limpn Jip Ruy(tt)dt — or 
1 po 1 po 
Pxy =Rxy(t)|t =0O0=- Ryy(0) Also Pxy -_ = J oxy) da and Pyx = = | yx Gigs 


Properties of cross power density spectrum: The properties of the cross power for real 
random processes X(t) and Y(t) are given by 


(1) Sxy(-@)= Sxy(@) and Syx(-@)= Syx(@) 


Proof: Consider the cross correlation function Rxy(t). The cross power density spectrum is 


Sxy(@) = [7 Ryy@ e- ior ar 

Let t=-t Then 

Sxy(@) = fo RyyCo eior g, Since Rxy(-t) = Rxy() 

Sxy(@) = f° Ryy@ e-i-wt gy 

Therefore Sxy(-@)= Sxy(@) Similarly Syx(-@)= Syx(@) hence proved. 


(2) The real part of Sxy(@) and real part Syx(@) are even functions of @ i.e. 
Re [Sxy(@)] and Re [Syx(@)] are even functions. 


Proof: We know that Sxy(@) = J. R -jwt g, and also we know that 


xYM e 
‘eats oe co 
e J°T=cosat-jsinat, Re [Sxy(@)] = I. Ryd cosat ac 


Since cos wt is an even function i.e. cos wt = cos (-at) 
Re [Sxy(@)] = ie Ry@ coswt dt ee Ry@ cos(—at) dt 
Therefore Sxy(@)= Sxy(-@) Similarly Syx(@)= Syx(-@) hence proved. 


(3) The imaginary part of Sxy(@) and imaginary part Syx(@) are odd functions of @ i.e. 
Im [Sxy(@)] and Im [Syx(@)] are odd functions. 


Proof: We know that Sxy(@) = J. R and also we know that 


XY) e-JOT gr 
e JT=cosont-jsinat, 
Im [Sxy(@)] = ie Ryy@(—sinat) dt ~~ ie Ryy@sinot ar = ~ 1M [Sxyv(@)] 


Therefore Im [Sxy()] = - Im [Sxy(@)] Similarly Im [Syx(@)] = - Im [Syx(@)] hence proved. 


(4) Sxy(@)=0 and Syx(w)=0 if X(t) and Y(t) are Orthogonal. 


Proof: From the properties of cross correlation function, We know that the random processes 
X(t) and Y(t) are said to be orthogonal if their cross correlation function is zero. 


Le. Rxy(t) = Ryx(t) =0. 
We know that Sxy(@) =f) Ryy@ pier gq 
Therefore Sxy(@)=0. Similarly Syx(@)=0 hence proved. 
(5) If X(t) and Y(t) are uncorrelated and have mean values X and Y, then 
Sxy()=27X Y 5(w). 
Proof: We know that Sxy(@) =f", Ryy@ ¢-iot gr 
= Sxy(o) = [7 E[X@Y (e+ dle" dr 
Since X(t) and Y(t) are uncorrelated, we know that 
E[X()Y(t+7)=E[X(QJE[Y(t + 7)] 
Therefore Sxy(@) = f- E[X(t)]E[Y(t + t)]e J" dr 
Sxy(@) = XY eJ°? dr 
Sxy(@) =X ¥ fe J? dr 


Therefore Sxy(@)=27X Y 6(w). hence proved. 


UNIT 5 


STOCHASTIC PROCESSES—SPECTRAL CHARACTERISTICS 
Definition of Power Spectral Density of a WSS Process 


{4 (2)} 


Let us define the truncated random process as follows 


XpQ= XY -T<t<T 


=0 otherwise 


2 X@rect(—) 


rect es 
where 27 is the unity-amplitude rectangular pulse of width 2r centering the origin. As 


Eom tan) will represent the random process {x} define the mean-square integral 


it 
FTX, (@) = [Aras 


Applying the Pareseval's theorem we find the energy of the signal 


r © 
[#7004 - [lFrXr(afao 


Therefore, the power associated with {Xp} is 


2 
oo ie 

sq [Ar@et == [|FTX,(@)| do 

or J 2T J i 


The average power is given by 


Fee | = sph fms | d@= pp PRON ao 


E|FTX,(a)/ 


Where ar the contribution to the average is power at frequency w and represents the 
power spectral 


density of {47} . As f | the left-hand side in the above expression represents the 
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average power of {x} 


Therefore, the PSD sxe) of the process {x} is defined in the limiting sense by 


_ B|FTY,(a)l 
S4(0)= fin =P 


Relation between the autocorrelation function and PSD: Wiener-Khinchin-Einstein 
theorem 


We have 
zl PEX,(@) P 


PTX, (@) FTX, (a) 
aT 


=H 


Figure 1 


Note that the above integral is to be performed on a square region bounded by 4 and 


= + — = = 
eal as illustrated in Figure 1.Substitute 414 that 47s a family of straight 
lines parallel to aor 
(27-|t at. 


‘The differential area in terms of * is given by the shaded area and 


equal to The double integral is now replaced by a single integral in * 
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Therefore, 


pETX, @X, ‘(@) _ 1 


ir Meee 
ae cogil R, (De *(2T-|thdt 
=f Roe! har 

ir * 2T 


” i Ry (der de 
if Rx(®) is integral then the right hand integral converges to -© as f 0 


: tim ZPEECOh [ Re(der"ae 
Pam 2r 0 


| FTX, (a) 


‘ee GF “isthe 


As we have noted earlier, the power spectral density 
contribution to the average 


power at frequency “ and is called the power spectral density of {x@)} . Thus , 


S,(@) = [Re (De dr 


and using the inverse Fourier transform 


Ry (0D) = 2s f Sy (Meda 
on + 


Example 1 The autocorrelation function of a WSS process {x} is given by 
Ry =ate Kl 2 >0 


Find the power spectral density of the process. 


S,-(@) = J Re ar 


co =p _ i 
J a%e Hl, JOU 
= | atets JOT ges fate OT JOT 7, 
aoe 0 
a? a” 
= + 
b-j@ b+ ja 
_ 2a%b 
hb? +a 


The autocorrelation function and the PSD are shown in Figure 2 
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Wiener-Khinchin-Einstein theorem 


The Wiener-Khinchin-Einstein theorem is also valid for discrete-time random processes. The 


power spectral density Sx(@) of the WSS process {X[»]} is the discrete-time Fourier transform 
of autocorrelation sequence. 


Sy(a)= = Ry[m]eo™ -KSweSn 


Mew 


R, [m1] J, related to Sx(@) by the inverse discrete-time Fourier transform and given by 


Sy(@e""d @ 


Thus *: 2] and Sx\@) forms a discrete-time Fourier transform pair. A generalized PSD can 
be defined in terms of 2 ~ transform as follows 
Sy@)= > &,[]z™ 
Me—D 


clearly, 


Syl a) = Sy(Z)L ie 


Linear time-invariant systems 


In many applications, physical systems are modeled as linear time-invariant (LTI) systems. 
The dynamic behavior of an LTI system to deterministic inputs is described by linear differential 
equations. We are familiar with time and transform domain (such as Laplace transform and 
Fourier transform) techniques to solve these differential equations. In this lecture, we develop 
the technique to analyze the response of an LTI system to WSS random process. 


The purpose of this study is two-folds: 


Analysis of the response of a system 

Finding an LTI system that can optimally estimate an unobserved random process from 
an observed process. The observed random process is statistically related to the 
unobserved random process. For example, we may have to find LTI system (also called a 
filter) to estimate the signal from the noisy observations. 


Basics of Linear Time Invariant Systems 
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A system is modeled by a transformation T that maps an input signal x(e) to an output 
signal y(t) as shown in Figure |. We can thus write, 


y@®) = T[x@)] 


Figure 1 


Linear system 


The system is called linear if the principle of superposition applies: the weighted sum of 
inputs results in the weighted sum of the corresponding outputs. Thus for a linear system 


¥ [ax (t)+a,x, (¢ )] = aT | % (t)] +a,T [> (t J] 
Example 1 Consider the output of a differentiator, given by 


__ ax(t) 
y® re 
a 
ee al aX, (£) + aX, (E) ) 


d a 
mn 47) a 413 () 


Hence the linear differentiator is a linear system. 
Linear time-invariant system 

Consider a linear system with y (t) = Tx (t). The system is called time-invariant if 
Tx(t-h) =ylt-h) V % 


It is easy to check that that the differentiator in the above example is a linear time - 
invariant system. 
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Response of a linear time-invariant system to deterministic input 


As shown in Figure 2, a linear system can be characterised by its impulse response 
h(t) = T(t) where S(£) is the Dirac delta function. 


system 


Figure 2 


Recall that any function x(t) can be represented in terms of the Dirac delta function as follows 


© 


x(t) = fx d(t-s) ds 


—o. 


If x(t) is input to the linear system y (¢) = Tx (t), then 


ye) r fx ra) (¢ - s) ds 


fr© TS (é 7 s) ads [ Using the linearity property | 


ss 


fx© h(t,s) ds 


—~o 


_#(ts) = T4(t-s) 


is the response at time ¢ due to the shifted impulse? b(t-s) 


If the system is time invariant, 
h(t,s) = k(t-s) 
Therefore for a linear-time invariant system, 
o 


y(t) = J x(s) h(t-s) ds = x()*2© 


th 
where * denotes the convolution operation. 


We also note that x(t) xe) = AZ) *x(Z). 
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Thus for a LTI System, 
yt) = x(t) * R(t) = R(t) *x(Z) 


Taking the Fourier transform, we get 


where H(a@) =FTh (t) = feo e? dtis the frequency response of the system 


Figure 3 shows the input-output relationship of an LTI system in terms of the impulse response 
and the frequency response. 


° 7 
Att) 

“e . 
A (ea) 


Figure 3 
Response of an LTI System to WSS input 


Consider an LTI system with impulse response h (t). Suppose {XO} ig a WSS process 
input to the system. The output @} of the system is given by 


{h(s) X(e-8) de= pales) X(6) as 


0 


Where we have assumed that the integrals exist in the mean square sense. 
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Mean and autocorrelation of the output process (FO) 


BY (t)= B [ h(6)X(t-s)as 


0 


F/) (s)BX (¢ ae s)ds 


Where #(0) is the frequency response at 0 frequency (@ = 0 ) and given by 


H(@)|, = preorea = flea 


T The Cross correlation of the input {X(t)} and the out put /Y (¢)} is given by 


Lia] 


£(x(t+r)¥ (t= £ ¥(t+r) J as) ¥(t-s) as 


Ats) # X(t+c) X(t-s) ds 


h(s) Ry (e+ sjds 


h(-w) Ry (t-w)de 


-t) * R(t) 


feed ny i—— 46 


)* Rylr) 
wl-t)= h(t) * Ry(-7) 
= A(t) * Ry (tr) 


Therefore, the mean of the output process O} is a constant 


The Cross correlation of the input {X (t)} and the out put /Y (t)} is given by 
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hfs) £ X(t+r) X{(t-s) as 


h(s) Ry le+sjds 


A——5 f—o fas 


h{-w) Ry (r—-w)de 


)* R(t) 


ll 

ox 
ae 

ry 


R w(t) = Al-r) * Ryle) 
aiso Ryle) = Ryf-r)= Alt) * Rel -z) 
= Alt) * Ry(z) 


Thus we see that Ra (Fig a function of lag * only. Therefore, {4@)} and @} are jointly 


wide-sense stationary. 


The autocorrelation function of the output process {Y(t)} is given by, 
h(s) X(¢tr-s)ds¥@) 


fee) EX(tt+r-s) Y(t) ds 


oO 


pals) R wf{t -s) ds 


h(t) * Rp{t) = ble) *h(-t) *R fr) 


Thus the autocorrelation of the output process FO} depends on the time-lag t ie., 


RY (t)¥ (t+2)= Ry (r) 


Ry (t) = Ry (e)*h (rt) *h(-1) 


The above analysis indicates that for an LTI system with WSS input 
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e the output is WSS and 
e The input and output are jointly WSS. 


The average power of the output process @} is given by 


Power spectrum of the output process 


Using the property of Fourier transform, we get the power spectral density of the output 
process given by 


Sy(@)H(@)#' (a) 
Sy (w)|# (@)f 
Also note that 


R(t) = A(-t) * Rye) 
and Ry(t) = h(r) * Ry(r) 


Taking the Fourier transform of Ry (2) we get the cross power spectral density 
Sy (@) 


given by 
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Figure 4 


Example 3 


A random voltage modeled by a white noise process ite (¢ ) 

No 
2 is input to an RC network shown in the Figure 7. 
Find (a) output PSD *r (®) 


: . & 
(b) output auto correlation function ~* (=) 


2 
(c) average output power ss (4) 


Xt) Cc * Y(t} 


Figure 7 


The frequency response of the system is given by 
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with power spectral density 


Therefore, 


ita +] Sy (a } 
1 ®, 


Pe +1 2 


(b) Taking the inverse Fourier transform 


(c) Average output power 


iM, 
HY? (t) = Ry, (0)=—* 
(¢) = % (0) = 2 
Rice's representation or quadrature representation of a WSS process 


An arbitrary zero-mean WSS process {4} can be represented in terms of the slowly 


£6) ang © 


varying components and as follows: 


Alt) = AX, (6) cos ae — A, (£) sin aye (1) 


B 
%-Fslalsa*D) x 
A (£). 


where “is a center frequency arbitrary chosen in the band and 


#,© are respectively called the in-phase and the quadrature-phase components of 


Let us choose a dual process (F@)} such that 
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X)+ JV O =(4,0 + i, Oj)e"" 


= (40) cos apt — A, (2) sin aye ) + ij (4,(2) sin apit X,(f) cos ape) 
— ——_— 
Zit Fit) 


then , 


A(t) = X(t) cos at + P(e) sin ape (2) 


and 


A, (t) = X(t) cos at —¥ (£2) sin ane (3) 


For such a representation, we require the processes {£,0)} and {£,0)} to be WSS. 


Note that 


EX (t) = cos a f#X, (f) — sin atZX, (t) 


As {£4} is zero Mean, we require that 
#X,(t) =0 


EX, (t) =0 
Again 


EX (t) =cosapt ZX (t)+ sin at ZY (t) 
EX ,(t) = cos tZX (t)— sin aytF Y(t) 


As each of £¥Y,(t), £X,(t) and #X(t) is zero-mean, we require that 
#Y@=0. 


Also 
Ry (¢+7,t)= EX (t+) cos ay (t+ T)+ ¥(t+ r)sin a, (t+r)[X@) cos at + Y(t) sin apt] 


=Ay(t) cosa, (t+ tT) cosapt+ Ry (r)sin a, (¢ + t) sin at + Ky, (tT) cos a, (t+ T) sin aye 
+ Ryy(T) sin a, (t+ 7) cos apt 


and 


Ry, + 7,0) =Ry(r) cos aye +r) cos age + Ry (T)sin ay (é + t)sin ape 
— Ry (r) cos ay (£ +7) sin ay — Ry (¢) sin a (¢ +7) cos ape 
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and 
Axx, tt.) =Ay(ticos Ge t+ Tcos ae — Ry (Tysin a E+ T) sin aye 
-R,, (tT) cos a, + t)sin e+ A, (sin GE + T) cos ae 


Thus, Ay (+4,2),Ry +7.) and Ay y @+7,2) will be independent of t if and only if 


Ry x, (7) = Ry(e) cos a(t + 7) cos ape — Ry(r) sin ay ( +7) sin ane 
— Ry (tr) cos a(t +7) sin apt + Ky (tr) sin ay (f +t) cos ape 
= Ry(r)[cos a, (é +7) cos at — sin ay (f +7) sin ape] 


— Ry (4) [cos a (f + 7) sin at — st n ay (¢ +7) cos anf] 
=Ry(t)cosayr — Ry, (4) sin(-a,r) 
=Ry(r)cosayr — Ry(r) sin ayr 


How to find (F@)} satisfying the above two conditions? 


For this, consider (7 @)} to be the Hilbert transform of {40} , 1.e. 


Y(t) = [Xone - s)ds 


Where £ and the integral is defined in the mean-square sense. See the illustration 


in Figure 2. 
rt 


Figure 2 


The frequency response H(o) of the Hilbert transform is given by 
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if @=0 
if @<0 
if @=0 


| A Car) = — son (ar) 
and |H(@)|' =1 
“8, (a) =|H (a) S,(@) = Sy (e) 


WS(@), for @>0 

Syqp(@) = H (@) Sq, (@) = 
Aad a eo. for a< 0 
— JS yy (a), for @>0 
iS (@), for @<0 


Sy(@) =H’ (@)Sy,(@) “| 


The Hilbert transform of Y(¢) satisfies the following spectral relations 


Sy(@)= Sz (2) 
and 
Sy (@) = Sy (a) 


From the above two relations, we get 


Ry (t) = Ry (7) 
and 


Ry (=- x (T) 


The Hilbert transform of A (0) is generally denoted as A (f). Therefore, from (2) and (3) we 
establish 


Xalt) = X(t)cos at + XU) sin aye, 
X_(t)=X@)cos apt - X(#) sin a 
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Alf) = A,(£) 008 ae — Xa lt) sin ape 


The realization for the in phase and the quadrature phase components is shown in Figure 
3 below. 


Xcft) 


Hilbert 
fransforve 


| sine 


Figure 3 


From the above analysis, we can summarize the following expressions for the autocorrelation 
functions 


Ry (t= 2yD) 
= Ry(ticos att Ay (tT) sin aT 
= Ry (tcos att AH * AR, (TF) sin ae “ Rigel FY = BCE * Ry CTD 


= Ry (t)cos at + KR, (Hsin HT 


&, (0) = Hilbert transform of 2, (0) 


= [= g(t -aids 
1 7s 


See the illustration in Figure 4 


TT 
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The variances OK, and OK, are given by 


oy, = Oy = Ry (0). 


Ke 


Ry (e) and Ry, (r), 


Taking the Fourier transform of we get 


Sy(@- @)+ Sy (@+ a) la] < B 


0 otherwise 


Sy, (@) = Sy, (@) -| 


Similarly, 


=RK, (r)sing,r — Ry (t)cos ay, 
=R,(r)sina,t — RY (tr) cos ayr 


and 


ai (a) = 
Hots 0 otherwise 


| llialialilies a] <3 


(@) 


Notice that the cross power spectral density Soke 
Sy(@) 


is purely imaginary. Particularly, if 


is locally symmetric about ™ 

Syx, (@) =9 
Implying that 
Ry. x, (F) =0 


(£,0) ang (XO) 


Consequently, the zero-mean processes are also uncorrelated 
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PROBABILITY THEORY AND STOCHASTIC PROCESSES 
Unitwise Important Questions 
UNIT-I 
Two marks Questions: 


. Define Demorgans’ laws. 

. Give the classical definition of Probability. 

. Define probability using the axiomatic approach. 

. Write the statement of multiplication theorem. 

. What are the conditions for a function to be a random variable? 
. Give the statement of total probability theorem. 

. Define countable and uncountable set? 

. Define universal set with properties? 

. Define Bernoulli trials? 

10. Define experiment with example? 

11. Give the Statement of multiplication theorem? 

12. Define independent events in case of two events A and B? 


OANINHNMN HB WN 


13. Define Venn diagram and represent joint P(ANB) in Venn diagram? 

14. Define complement laws of sets? 

15. Is it possible to define a Continuous random variable on discrete sample space? If , yes give 
example. 

Three Marks Questions: 


1. Define sample space and classify the types of sample space. 

2. Define Joint and Conditional Probability. 

3. Define equally likely events, Exhaustive events and mutually exclusive events. 

4. Show that P(AUB)=P(A)+P(B)-P(ANMB). 

5. Define Random variable and Write the classifications of Random variable. 

6. In the experiment of tossing a dice, what is the probability of face having 3 dots or 6 dots to 
appear? 

7. Define Probability axioms? 

8. For any two events A and B in a Sample space S such that B is a subset of A then Show that 
P(A\B)=1. 

9. Define the properties of probability? 

10. Determine the probability of the card being either red or a king when one card is drawn from 
a regular deck of 52 cards. 

11. Define complement of a set and show that P(A')=1-P(A) 

12. Two cards are selected from 6 cards numbered from | to 6 . Find the probability that the sum 
is odd if two cards are drawn together. 

13. Explain the concept of total probability with neat Venn diagram. 


14. What is the probability of selecting two ace cards from a deck of 52 cards? 
15. If the sample space has n elements then show that it has 2” subsets. 


Ten Marks Questions: 


1.a) State and Prove Bayes’ theorem. 
b) Write the Mathematical model of experiment. 


2. In a box there are 100 resistors having resistance and tolerance values given in table. Let a 
resistor be selected from the box and assume that each resistor has the same likelihood of being 
chosen. Event A: Draw a 47Q resistor, Event B: Draw a resistor with 5% tolerance, Event C: 
Draw a 100Q resistor. Find the individual, joint and conditional probabilities. 


Resistance Tolerance Total 
(Q) 5% 10% 

22 10 14 24 
47 28 16 44 
100 24 8 32 
Total 62 38 100 


3. a) Two boxes are selected randomly. The first box contains 2 white balls and 3 black balls. 
The second box contains 3 white and 4 black balls. What is the probability of drawing a white 
ball. 

b) An aircraft is used to fire at a target. It will be successful if 2 or more bombs hit the target. If 
the aircraft fires 3 bombs and the probability of the bomb hitting the target is 0.4, then what is 
the probability that the target is hit? 


4. a) An experiment consists of observing the sum of the outcomes when two fair dice are 
thrown. Find the probability that the sum is 7 and find the probability that the sum is greater than 
10. 

b) In a factory there are 4 machines produce 10%,20%,30%,40% of an items respectively. The 
defective items produced by each machine are 5%,4%,3% and 2% respectively. Now an item is 
selected which is to be defective, what is the probability it being from the 2™ machine. And also 
write the statement of total probability theorem? 


5. Determine probabilities of system error and correct system transmission of symbols for an 
elementary binary communication system shown in below figure consisting of a transmitter that 
sends one of two possible symbols (a 1 or a 0) over a channel to a receiver. The channel 
occasionally causes errors to occur so that a ’1’ show up at the receiver as a ’0? and vice versa. 
Assume the symbols ‘1’ and ‘0’ are selected for a transmission as 0.6 and 0.4 respectively. 


P(B1)=0.6= 


P(B:)=0.4 


6. In a binary communication system, the errors occur with a probability of “p”, In a block of 
“n” bits transmitted, what is the probability of receiving 

i) at the most 1 bit in error 

1i at least 4 bits in error 

\ 

7. Let A and B are events in a sample space S. Show that if A and B are independent, then so are 
a) AandB b) A andBc)A andB 


8.a) An experiment consist of rolling a single die. Two events are defined as A = { a 6 shows 
up}: and B= {a2 or a5 shows up} 

i) Find P(A) and P(B) 

ii) Define a third event C so that P(C) = 1-P(A)-P(B) 

b) The six sides of a fair die are numbered from t to 6. The die is rolled 4 times. How many 
sequences of the four resulting numbers are possible? 


9.a) State and prove the total probability theorem? 
b) Explain about conditional probability. 


10.In the experiment of tossing a die, all the even numbers are equally likely to appear and 
similarly the odd numbers. An odd number occurs thrice more frequently than an even number. 
Find the probability that 

a) an even number appears 

b) a prime number appears 

c) an odd numbers appears 

d) an odd prime number appears. 


11. 1.The probability mass function of random variable X is defined as P(X=0)=3C’, 
P(X=1)=4C-10C?,P(X=2)=5C-1,where C>0 and P(X=r)=0 if r40,1,2 find 
a)The value of C b) P(O<X<2/X>0) 


12.a)A sample space contains 3 sample points with associated probabilities given by 2p,p” and 
4p-1. Find the values of p. 
b)If P(A)=0.4,P(B)=0.7 and P(ANB)=0.3 find P(A'NB’). 


13.a)Two fair dice are thrown independently. Three events A,B,C are respectively defined as 
follows: i)Odd face with first die — 11) Odd face with second die i1i)The sum of two numbers in 
the two dice is odd. Are the events A, B, C mutually independent or pair wise independent? 

b) Distinguish between conditional and unconditional probabilities with example. 


5 
14. a) If P(A+B) = a P(AB)=1/8 and P(B')=1/2. Prove that the events A and B are 


independent. 

b) A pack contains 6 white and 4 green pencils; another pack contains 5 white and 7 green 
pencils. If one pencil is drawn from each pack, find the probability that 1) both are white 

1i) One is white and another is green. 


15) Urn A contains 5 red marbles and 3 white marbles ,urn B contains 2 red marbles and 6 white 
marbles . (a) If a marble is drawn from each urn, what is the probability that they 

are both of the same color? (b)If two marbles are drawn from each urn, what is the probability 
that all four marbles are of same color? 


UNIT-II 
Two marks Questions: 


. Define Probability density and distribution function. 

. Define the expected value of Discrete Random variable and Continuous Random Variable. 
. Define Moment generating function and Characteristic Function of a Random variable. 

. Define moments about origin and central moments. 

. Show that Var(kX)=k? var(X), here k is a constant. 

. Define skew and coefficient of skewness. 

. Find the Moment generating function of two independent Random variables X; & X2. 

. Write the statement of Chebychev’s inequality. 

. Define variance of a Discrete random variable. 

10. Show that E(KX)=KE(X). 

11. Show that Fx(x)=1-P(X>x). 

12. Draw the distribution and density curves of Gaussian random variable. 

13. The mean and variance of the binomial distribution are 4 and 3 respectively. Find P(X=0). 
14. Find the mean of uniform density function. 

15. Find the maximum value of Gaussian density function. 
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Three marks Questions: 


1. Derive the expression for the density function of Discrete Random variable. 
2. Find the variance of X for uniform density function. 

3. Define various types of transformation of Random variables. 

4. Write the properties of Gaussian density curve. 

5. Find the moment generating function of binomial distribution 


6. In an experiment when two dice are thrown simultaneously, find expected value of the sum of 
number of points on them. 

7. Derive the expression for distribution function of uniform Random variable. 

8. Define moment generating function and write the formula to find mean and variance? 

9. The mean and variance of the binomial distribution are 4 and 3 respectively. Find P(X=0) ? 
10. Is the function defined as follows a density by function? 


0 x<2 
f(x)= 4 —(3+2x)  2<x<4 
0 x>4 


11.In which probability distribution, variance and mean are equal and define corresponding 
distribution ? 

12. If the random variable X has the following probability distribution 

Xx: -2 -1 0 1 

P(x): 0.4 K 0.2 0.3. Find k and the mean of x. 

13. Derive the expression for variance in terms of first and second moments. 

14. Show that Fx(-00)=0 and Fx(co)=1 


15.If the MGF of a uniform distribution for a random variable X is ~ (e°” — e*”) Find the 
E[X]? 


Ten Marks Questions: 


1.a) The exponential density function given by 
fx(x) =(1/b)e x2o 
=0 x<a 
Find the mean and variance. 
b) Define Moment Generating Function and write any two properties. 


2. Derive the Binomial density function and find mean & variance. 
3. Derive the Poisson density function and find mean & variance 


4. If X is a discrete random variable with a Moment generating function of M;(v), find the 
Moment generating function of 


i) Y=aX+b ii) Y=KX iii) y= 


5. Arandom variable X has the distribution function 


12 05 
Fx(x)= >. u(x —n) 
* n= 030 
Find the probability of a) P{-0<X<6.5} b)p{X>4} c) p{6< X < 9} 


6.a) Let X_be a Continuous random variable with density function 
f= JS+K 0x6 
0 otherwise 
Find the value of K and also find P{2 < X < 5} 
b) The probability density function of a random variable X is given by f(x) = for -3<x<6 and 


equal to zero otherwise. Find the density function of Y==(12-x) 


7. a) Verify the Characteristic function of a random variable is having its maximum magnitude at 
w=0 and find its maximum value. 
b) Find the Moment generating function of exponential distribution? 


8. a)Write short notes on Gaussian distribution and also find its mean? 
b) Consider that a fair coin is tossed 3 times, Let X be a random variable, defined as 
X= number of tails appeared, find the expected value of X. 


9 .a) State and prove the chebychev’s inequality theorem? 
b) Find the probability of getting a total of 5, at-least once in 4 tosses of a pair of fair dice. 


10a) If X is uniformly distributed over (0,10) calculate the probability that 3<x<8. 
b)If the probability density function of X is fx(x) = 2x, O<x<1, find the probability density 
function of Y=3X+1. 


11a). If X and Y are independent poisson variates such that P[X = 1]=P[X=2] and 
P[Y=2] = P[Y=3] find v[X-2Y]. 
b) Let X be a continuous random variable with pdf f (x)= { 3x O<x<l 
0 else find P(X<0.6)? 


12. Find the moment generating function of a random variable x having the probability density 
function 


NI& 


f(x) = Je” x>0 


0) else 


and hence find the mean and variance. 


13.a)If X is uniformly distributed is [-2,2], find (i) P(x<0) 1)P( IX -11)= =) 
b) If X is uniformly distributed with E(x)=1 and Var(x)=4/3, find P(X<0) 


14.A random variable X has the following probability density function 


x O<x<l1 
iy k(2-x) 1<x<2 
0) else 


i) Find the value of k? 
ii) Find P(0.2<x<1.2)? 
1i1) Find the distribution function of X. 


15.a)Derive the m.g.f of Poisson distribution and hence or otherwise deduce its mean and 
variance. 
b)If the probability density of X is given by f(x)=| 2(1-x) O<x<l 
0 else 
Find its r" moment and evaluate E[(2X+1)*] 


UNIT-III 
Two marks Questions: 


. Define the statistical Independence of the Random variables. 

. Define point conditioning & interval conditioning distribution function. 
. Give the statement of central limit theorem. 

. Define correlation and covariance of two random variables X& Y. 

. Define the joint Gaussian density function of two random variables. 

. Define marginal distribution and density functions. 

. Define joint density function of two random variables. 

. Give the statement of central limit theorem. 

. Define the joint moments about the origin of two RVS X and Y. 

10. Define orthogonal random variables. 


OANINHNDN FWY 


11. Define expected value of a function of two and N random variables. 
12. Define two joint central moments for two-dimensional random variables X and Y. 
13. Define marginal characteristic functions. 
14. Define correlation coefficient with two properties. 
15. Define joint moment generating function and write its expression for continuous and discrete 
RVS. 
Three Marks Questions: 


1. If E[X]=2, E[Y]=3, E[XY]=10, E[X*]=9, and E[Y?]=16 then find variance & covariance of 
X&Y. 
2. The joint probability density function of X&Y is 
fx y(x,y)= { c(2x+y); 0<x<2,0<y<3 
0; else Then find the value of constant c. 

3.If X and Y are two independent RVs such that E(X)=A1,variance of X=07,E(Y)= Ao, and 
variance of Y=o?, Show that var(XY)=07 of + Ato? + 1307. 
4. Show that var(X+Y) = var(x)+var(Y), if X&Y are statistical independent random variables. 
5. Show that Fx y(x,00)=Fx(x) and Fx,y(©, y)=Fy(y) 
6. The joint probability density function of the random variable (X,Y) is given by 

f(x,y)=Kxy e~@*+¥") _x>0, y>0. Find the value of K. 
7. If two random variables X and Y have p.d.f f(x,y) = k(2x+y) for 0<x<2, 0<y<3, evaluate k. 
8. Find the correlation co-efficient of two independent RVS X and Y. 
9. If X and Y are two RVs then derive the expression for covariance in terms of correlation. 
10. Show that ®y y (w1, w2) = y(w1)hy(w2) if X and Y are independent RVs. 
11. If X and Y are independent random variables, show that E[XY] =E[X].E[Y] 
12. Explain when random variables are called jointly random Gaussian, and also list important 
properties of Gaussian random variables? 
13. If X is a random variable for which E(X) =10 and var(X) =25 , for what possible value of ‘a’ 
and ‘b’ does Y=aX-b have exception 0 and variance | ? 


14. Stastistically independent random variables X and Y have moments mj0=2, m20=14, and 
my1= -6.Find the moment 122. 
15. Show that cov(X+a, ¥Y+b)=cov(X, Y)=Cxy. Where a and b are constants. 


Ten Marks Questions: 


1. a) State and prove the density function of sum of two random variables. 
b) The joint density function of two random variables X and Y is 
(ty)? 


feray) =2 ap TE <* <land—-3<y<3 


0; otherwise 


Find the variances of X and Y. 
2. a) Let Z=X+Y-C, where X and Y are independent random variables with variance 07x, o”y 
and C is constant. Find the variance of Z in terms of 07x, o’y and C. 
b) State and prove any three properties of joint characteristic function. 


3.a) State and explain the properties of joint density function 
b) The joint density function of random variables X and Y is 


_ (8xy; O<x<10<y<1 
fry Gy) = 0, otherwise 
Find f(y/x) and f(x/y) 


4. The input to a binary communication system is a random variable X, takes on one of two 
values 0 and 1, with probabilities %4 and 4 respectively. Due to the errors caused by the channel 
noise, the output random variable Y, differs from the Input X occasionally. The behavior of the 


communication system is modeled by the conditional probabilities P(—) = - and P (=) = A 


Find a) The probability for a transmitted message to be received as 0 
b) Probability that the transmitted message is al. If the received is a 1. 


5. Let X and Y be the random variables defined as X=Cos@ and Y=Sin0 where @ is a uniform 
random variable over (0, 270) 

a) Are X and Y Uncorrelated? 

b) Are X and Y Independent? 


6. a) Define and State the properties of joint cumulative distribution function of two random 


variables X and Y. 
b) A joint probability density function is fr y(X,y ) = a 0<x<60<y<4 


0 else where 
Find the expected value of the function g(X, Y)= (xy 


7. State and prove the central limit theorem. 


8. Two random variables X and Y have zero mean and variance of = 16 and of = 36 
correlation coefficient is 0.5 determine the following 

i) The variance of the sum of X and Y 

il) The variance of the difference of X and Y 


9. A certain binary system transmits two binary states X = +1 and X = -1 with equal probability. 
There are three possible states with the receiver, such as Y = +1, 0 and -1. The performance of 
the communication system is given as 
P(y = +1/X = +1) =0.2; P(Y= +1/X= -1) = 0.1; P(Y = 0/X = +1) = P(Y = 0/X = -1) = 0.05. 
Find a) P(Y = 0) 

b) PX =+41/Y = +1) 

c) P(X =-1/Y =0). 


10. Two random variables X and Y have the joint pdf is 


ued AeeY) x,y>0 
0 elsewhere 


i. Evaluate A ii. Find the marginal pdf's 

ili. Find the marginal pdf’s 

iv. Find the joint cdf 

v. Find the distribution functions and conditional cdf’s. 


11. Given is the joint distribution of X and Y 


xX 
0 1 2 
0.02 | 0.08 | 0.10 
Y 1 | 0.05 | 0.20 | 0.25 
0.03 | 0.12 | 0.15 


Obtain (i) Marginal Distributions and (ii) The Conditional Distribution of X given Y=0. 


12. The joint probability mass functions of X and Y is given as 


X 
P(x,y) 0 1 ) 


0.1 | 0.04 | 0.02 
0.08 | 0.20 | 0.06 
0.06 | 0.14 | 0.30 


x 


NI] | oO 


Compute the marginal PMF of X and Y,P[x<1, y<1] and check if X and Y are independent. 


13.The joint probability density function of the two random variables is 


(3.9) 7 Sxy I<x<y<2 
0 else 
i)Find the marginal density functions of X and Y 
(ii) Find the conditional density function of Y given X 
14. a) If the independent random variables X and Y have the variances 36 and 16 respectively, 
find the correlation coefficient between X+Y and X-Y 


b) If the joint PDF of X and Y is fx.y(x,y )={ e~%*t” x>0,y>0 check whether X and Y are 
0 else independent? 


15a) The regression equation of X on Y is 3Y-5x+108=0. If the mean value of Y is 44 and the 
variance of X is 9/16" of the variance of Y Find the mean value of X and the correlation 


coefficient. 
b) If X and Y are uncorrelated random variables with variances 16 and 9. Find the Correlation 


co-efficient between X+Y and X-Y. 


UNIT-IV 


Two marks Questions: 

. Define wide sense stationary random processes. 

. Give the statement of ergodic theorem. 

. Define the auto covariance & cross covariance functions of Random processes X(t). 
. When two random processes X(t)& Y(t) are said to be independent. 

. Define the cross correlation function between two random processes X(t) & Y(t). 

. Define time — invariant system. 

. Define Poisson Random process. 

. Define random process with example. 

9. Define time average of the random process X(t). 

10. Define orthogonal random processes. 

11. {X(s,t)} is a random process , what is the nature of X(s,t) when (a) s is fixed (b) t is fixed? 
(c) When s and t both are varying. 

12. Define strict sense stationary process 

13. Define stationary random process and list various levels of S.R.P. 

14. Give the statement of super position principal. 

15. Define strict sense stationary random process. 


CONN FSF WN 


Three Marks Questions: 


1. Differentiate between Random Processes and Random variables with example 

2. Prove that the Auto correlation function has maximum value at the origin i.e | Rxx(t) | = 
Rxx(0) 

3. A stationary ergodic random processes has the Auto correlation function with the periodic 


components as Rxx(t) = 25 + find mean of the random process. 


1+6t? 
4. Define mean ergodic random processes and correlation ergodic Random processes. 

5. Find the mean value of Response of a linear system. 

6.Find the mean of the stationary process X(t), whose autocorrelation function is given as 


R(t) =16 + — 
1+16T 
7. State any two properties of linear time invariant system. 
8. Prove that fora WSS RP X (t), Auto correlation is even function of T. 
9. Define continuous and discrete random process with examples. 
10. Define correlation matrix of two random processes X(t) and Y(t). 
11. Define jointly Ergodic random processes. 
12.If X(t) isa WSS random process with auto correlation function Rxx(t)=Ae~*% '*' determine 
the second order moment of the random variable { X(8)-X(5) }. 
13.Find the variance of the stationary process {X(t)} whose ACF is given by 


9 


1+61? 
14. Check whether the following functions are valid auto correlation functions. 


Rxx(t) = 16 + 


a) —— b) 2sin(mt) 


15.The auto correlation function for a stationary process X(t) is given by Rxx (t)=9 +2e 
Find the mean of the random variable Y = { X(t)dt. 


= )El 


Ten Marks Questions: 


1. a) Define Wide Sense Stationary Process and write it’s conditions. 
b) A random process is given as X(t) = At, where A is a uniformly distributed random variable 
on (0,2). Find whether X(t) is wide sense stationary or not. 


2. X(t) 1s a stationary random process with a mean of 3 and an auto correlation function of 
Rxx(t) = exp (-0.2 |x | ). Find the second central Moment of the random variable Y=Z-W, 
where ‘Z’ and ‘W’ are the samples of the random process at t=4 sec and t=8 sec respectively. 


3. Explain the following 

1) Stationarity 

i1) Ergodicity 

1ii) Statistical independence with respect to random processes 


4. a) Given the RP X(t) = A cos(wot) + B sin (wot) where wo is a constant, and A and B are 
uncorrelated Zero mean random variables having different density functions but the same 
variance o”. Show that X(t) is wide sense stationary. 

b) Define Covariance of the Random processes with any two properties. 


: : ; 6 sin(Tt . . 
5. a) A Gaussian RP has an auto correlation function RAGS Determine a covariance 
TT 
matrix for the Random variable X(t) 
b) Derive the expression for cross correlation function between the input and output of a LTI 
system. 
6. Explain about Poisson Random process and also find its mean and variance. 


7. The function of time Z(t) = Xicosq@ot- X2sinwot is a random process. If Xi and Xzare 
independent Gaussian random variables, each with zero mean and variance o7, find E[Z]. E[Z’] 
and var(z). 


8. Briefly explain the distribution and density functions in the context of stationary and 
independent random processes. 


9. Explain about the following random process 
(i) Mean ergodic process 

(ii) Correlation ergodic process 

(iii) Gaussian random process 


10. State and prove the auto correlation and cross correlation function properties. 


11. Examine whether the random process X(t)= A cos(@t+0) is a wide sense stationary if A and 
@ are constants and 0 is uniformly distributed random variable in (0,27). 


12. A random process X(t) defined by X(t)=A cos(t)+B sin (t) , where A and B are independent 
random variables each of which takes a value 2 with Probability 1/3 anda value | with 
probability 2 / 3. Show that X (t) is wide —sense stationary 


13.A random process has sample functions of the form X(t)= A cos(@t+0) where @ is 
constant, A is a random variable with mean zero and variance one and 0 is a random variable 
that is uniformly distributed between 0 and 2x. Assume that the random variables A and @ are 
independent. Is X(t) mean ergodic-process? 


14. If the WSS random process X(t) is given as X(t)=10cos(100t + 9) where 0 is a uniformly 
distributed over (-2,7).prove that X(t) is correlation ergodic. 


15.a)If X(t) and Y(t) are the random processes. with auto correlation functions Rxx(t) and 
Ryy(t) then show that IRxy(t) | < ./Ryx(0)Ryy(0) 
b) If the input to the LTI system is the WSS random process then show that output is also a 
WSS random process. 


UNIT-V 
Two marks Questions: 


. Define Power Spectrum Density. 

. Give the statement of Wiener-Khinchin relation. 

. Define spectrum Band width and RMS bandwidth. 

. Write any two properties of Power Spectrum Density. 

. Define a system. When is it called linear system? 

. Define Fourier and inverse Fourier transform of a deterministic signal X(t). 

. Define transfer function of the system. 

. Define unit impulse response of the system 

. What is the need of power density spectrum? 

10. Define energy and power of the random process X(t) within the time interval [-T,T]. 


2 
11. Sketch the PSD of the RP whose ACF is Rxx(=> SiN Wo T 


12. Define cross power between two random processes X(t) and Y(t). 

13. State any two properties of cross PSD. 

14. State the conditions for the existence of Fourier transform of any signal. 

15.If the Fourier transform of the RP X(t) is X(@) then find the Fourier transform of the RP 
X(t-to) . 

Three Marks Questions: 


OANINHNMN HB WN 


1. Show that Sxx(-@) = Sxx(@). 1.e., Power spectrum density is even function of o. 
2. If the Power spectrum density of x(t) is Sxx(@), find the PSD of “ x(t). 


3. If the Auto correlation function of wide sense stationary X(t) is Rxx(t)=4+2e?!7l, Find the 

area enclosed by the power spectrum density curve of X(t). 

4. Define linear system and derive the expression for output response. 

5. If X(t) & Y(t)are uncorrelated and have constant mean values X&Y then show that Sxx(@)= 

2T1XY6(w) 

6. Show that real part of Sxy(@) and Syx(@) are even functions of a. 

7. The power spectral density function of a wide sense stationary process is given by Sxx(@)= 
lw |<1 .Find the auto correlation function of the process. 


8. Find the power spectral density function of a stationary process if R(t) =e7 '7' 


9. Show that imaginary part of Sxy(@) and Syx(@) are odd functions of o. 
10. Find the variance of the stationary process {X(t)} where R(t)=2+4e~? '7! 

11. Show that Sxy(@)=Sxx(@)H(o). 

12. Show that Syx(@)=Sxx(@)H(-@). 

13. Give the statement of the relation between cross PSD and cross correlation function with 
corresponding mathematical expressions. 


14. Find the cross correlation function for the cross PSD Sxy(w) = so 


15. Differentiate between PSD and Cross PSD. 


Ten Marks Questions: 


1. a)Check the following power spectral density functions are valid or not 
. cos8(w) 
) 2+04 

b) Derive the relation between input PSD and output PSD of an LTI system 


ie 


2. Derive the relationship between cross-power spectral density and cross correlation function. 


3. A stationery random process X(t) has spectral density Sxx(w)=25/ (w?+25) and an 
independent stationary process Y(t) has the spectral density Syy(@)= w?/ (w?+25). If X(t) and 
Y(t) are of zero mean, find the: 

a) PSD of Z(t)=X(t) + Y(t) 

b) Cross spectral density of X(t) and Z(t) 


4. a) The input to an LTI system with impulse response h(t)= 6(t) + t?e~“*. U(t) is a WSS 
process with mean of 3. Find the mean of the output of the system. 
b) Define Power Spectral density with three properties. 
9 
w?+64 
Find 1) The average power of the process ii) The Auto correlation function 
b) State the properties of power spectral density 


5. a) A random process Y(t) has the power spectral density Syy(@)= 


6w : F 
ar Find the average power in 


6. a) A random process has the power density spectrum Syy(@)= 
the process. 


. : : ‘ 16 
b) Find the auto correlation function of the random process whose psd is Sad 


7. a) Find the cross correlation function corresponding to the cross power spectrum 


= 6 
Sx)" GyanGs jw)? 


b) Write short notes on cross power density spectrum. 


8. a) Consider a random process X(t)=cos(wt + 8)where w is a real constant and @is a uniform 
random variable in (0, 2/2). Find the average power in the process. 
b) Define and derive the expression for average power of Random process. 


9. a) The power spectrum density function of a stationary random process is given by 
Sxx(w)=J A, -K< w< K 
0, other wise 
Find the auto correlation function. 
b) Derive the expression for power spectrum density. 


10. a) Define and derive the expression for average cross power between two random process 
X(t) and Y(t). 


2 
b) Find the cross power spectral density for Rxx(=— sin(@oT) 


11. Find the power spectral density function of the stationary process whose autocorrelation 
function is given by e7 '*! 


12. X(t) and Y(t) are zero mean and stochastically independent random processes having 
autocorrelation functions Rxx(t)= e~ '*' and Ryy(t)= cos 2mT respectively. 

a)Find autocorrelation functions of W(t)=X(t)+Y(t) and Z(t)=X(t)-Y(t). 

b) Find cross correlation function between W (t) and Z (t) 


13.For a input —output linear system(X(t),h(t), Y(t)),derive the cross correlation function Rxy(t) 
and the output auto correlation functilon Rxx(t) 


14. Consider a system with transfer function eae .An input signal with auto correlation function 


m6(t)+m/? is fed to as input to the system. Find the mean and mean square value of the output. 


15. A random process is defined as Y (t) = X (t)-X (t-a), where X (t) is a WSS process and a > 0 is 
a constant. Find the PSD of Y (t) in terms of the corresponding quantities of X (t). 
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PART—A (25 Marks) 
1, (a) What is a random variable? Discuss various properties of a random variable 2 MARKS 
(b) Define the following 3 MARKS 
(i) Event (ii) Sample Space (iii) Conditional Probability 
(C) If the Random variable X has uniform density, find its mean and variance 2 MARKS 
(d) Define and explain the Gaussian random variable in brief? 3 MARKS 
(€) State and prove any four properties of characteristic function? 2 MARKS 
(f) Define moment generating function 3 MARKS 
(g) State the condition for wide sense stationary Random process 2 MARKS 
(h) Explain Variance Ergodic random process 3 MARKS 
(i) Explain the Relationship between Power Spectrum and Autocorrelation Function 2 MARKS 
(j) State any two properties of Power density spectrum 3 MARKS 
PART —B (50 Marks) 
SE —I 
2, 4&) Given that two events A and B are statistically independent, Show that {5) 


A and  , Band A are also independent . 
b) Determine the probability o f the card being either red or a king when one card [5] 
is drawn from a regular deck of 52 cards. 


(OR) 

ly launched if two relays A and B both have failed. The probabilities of 

A ‘ any 5 fle’ aes sorta be 0.01 and 0,03, respectively. It is also known that B is more likely 
to fail (probability 0,06) if A has failed. [5+5] 


1) What ability of an accidental missile launch? 
SWhatis the probability thet A il fall iB has fled? 


5] 


oe 4 a 6 
3K 


n them with their 


Scanned by CamScanner- 


Scanned by CamScanner 


R15 
Code No: R15A0403 
MALLA REDDY COLLEGE OF ENGINEERING & TECHNOLOGY 


(Autonomous Institution - UGC, Govt. of India) 
I! B.Tech | Semester supplementary Examinations, May 2017 
Probability Theory and Stochastic Process 


ae TTT rs 


Time: 3 hours Max. Marks: 75 
Note: This question paper contains two parts A and B 
Part A is compulsory which carriers 25 marks and Answer all questions. 
Part B Consists of 5 SECTIONS (One SECTION for each UNIT). Answer FIVE 
Questions, Choosing ONE Question from each SECTION and each Question carries 


10 marks. 
PART-A (25 Marks) 
1. (a) What is sample space? Explain the Discrete sample space and Continuous sample 
space each with a suitable example. [3M] 
(b) What is the probability of getting 53 Sundays in a leap year? [2M] 
(c) Explain Gaussian random Variable. [3M] 
(d) Define the moment generating function(MGF). [3M] 
(e) What is Central Limit theorem? [2M] 
(f) What is correlation co efficient and write its significance. [3M] 
(g) What is meant by Correlation ergodic process? [3M] 
(h) What is strict sense stationary process? [2M] 
(i) Write the properties of power density spectrum. [2M] 


() .Find the mean value of Random variable Y= -3X+22 where the mean of X is 3. [3M] 


PART -B (50 Marks) 
SECTION - I 
2.a)Define 1)ConditionalProbability(ii)Independentevents(iii)MutuallyExclusiveEvents 


(iv) Joint Probability 
b)When two dice are thrown find the probability of getting 
(i) {sum>7} (ii) {2<sum<=5 } iii) {sum>10} 
OR 


3. a) A batch of 50 items contains 10 defective items. Suppose 10 items are selected at 
random and tested. What is the probability that exactly 5 of the items tested are 
defective? 

b) State and prove Baye’s theorem. 


SECTION -— II 


4. a) Define probability density function and write its properties 


b) If random variable X is transformed to the new random variable Y= CX’, find the 
density function of Y. 


OR 


5.a) What is Probability distribution function? Explain its properties. 


b)A random variable has probability density function 
f(x) = c x(1-x) 0<x<]1 
0 else where 
Find i) ¢ ii. p(1/2<X<3/4) iii) Fy(x) 
SECTION - III 


6. a) Explain about joint moments & how they are generated. 

b) Mean values of random variables X ,Y are 0.5, 2. Mean square values are 1.5, 5.5 and 
covariance of X and Y is -0.25. Find cross correlation and correlation coefficient. 

OR 
7.a) Find the conditional density functions for the joint distribution 
Fx,y(x,y)=4xyexp(x"+y")u(x)u(y) 

b) Let X and Y be the random variables defined as X=cos 0 and Y=sin 0, where © is a 

uniform random variable over (0,27). Are X and Y uncorrelated? Are X and Y independent? 


SECTION — IV 
8. a) Write and prove properties of cross correlation function 


b) Find the mean and Auto correlation function of the Random process X(t) = Acos(wt+ 9) 
where A and w are constants, @ is random variable uniformly distributed on the interval 
(0,217) 


OR 
9. a) a) State and prove Wiener Khinchin theorem. 
b) X (t) is a WSS process and Y (t) = A cos (@ct + @) is a random process which is 
independent of X(t). Here, @ is a uniform random variable over (— x, 2). If the auto- 
correlation function of X (t) is Rxx(t), Find the auto correlation of Z(t)=X(t) Y(t). 


SECTION — V 


10.a) Derive the relationship between cross correlation function and cross power density 
spectrum. b) Find the power spectral density (P.S.D) of X(t)}=cos(wt+0) where @ is a 
uniform random variable over (0,2/2) and also find the average power. 


OR 
11. a) Find the relation between input and output power spectral density in a LTI System. 
b) Check whether the following functions are valid power spectral density (PSD) 
cos8w* 2 
. om —(w-1) 
) 2+@* il) € 
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Part A is compulsory which carriers 25 marks and Answer all questions. 
Part B Consists of 5 SECTIONS (One SECTION for each UNIT). Answer FIVE 
Questions, Choosing ONE Question from each SECTION and each Question carries 


10 marks. 
PART-A (25 Marks) 
1. (a) When do you say that two events are-independent? [2M] 
(b) A card is drawn from a pack of 52 cards. Find the probability of getting a king or a 
heart or a red card. [3M] 
(c) Discuss the conditions for a function to be a random variable. [2M] 
(d)-A discrete random variable X has possible values which occur with probabilities 0.4, 
0.25, 0.15, 0.1 and 0.1 respectively. Find the mean value X. , [3M] 
(e) Explain the physical significance of variance of a random variable. [2M] 
(f) Define joint central moments for the random variables X and Y. [2M] 
(g) What is meant by Mean Ergodic process? [3M]. 
(h) Explain the classification of random processes [3M] 
(i) Is power density spectrum an even function of ‘@’or odd function of ‘w’? Justify. [2M] 
(j) Find the mean value of exponential random variable. [3M] 
PART -B (50 Marks) 


SECTION -I 


2.a) Four cards are drawn from a well shuffled pack of playing cards. Find the probability 


that i) All are clubs ii) Two are spades & two are hearts iii) Four cards are from different 
suits. 


b) Show that Conditional probability satisfies the three axioms of probability 
OR 


3. a) Telephone calls are initiated through an exchange at the average rate of 75 per minute 
and are described by a Poisson process. Find the probability that more than 3 calls are 


initiated in any 5-second period. . 
b) Explain the terms Joint probability and Conditional probability. 
SECTION - il 


4. a) Determine E[X] and VAR[X] of Poisson random variable X. 


RI5 


b) List the properties of conditional density function & conditional distribution function. 
OR 
5. a) A’random variable has probability density function 
fx(x) =cx(l-x) O<x<l 
0 else where 
Find a) cb) P(1/2<X<3/4) c) Fx(x) 
b) Determine the cumulative distribution function and probability density function of Y 
given that Y = 2X+3 and that f(x) = 2e™ u(x). 


SECTION — IH 


6. a Define joint central moments for two random variables X and Y and explain the 
covariance of two random variables. 


b):The joint probability density function of two random variables X and Y is given by 
f(x,y) = c(2x+y) ; 0Sx<1,0<y<2 0; else where 


Find: i) The value of ‘c’. ii) Marginal distribution function of x and y. 


OR 


7. a) Derive the expressions for the distribution and density functions of sum of two 
statistically independent random variables. ° 


b) The density function of two random variables X and Y is 
fxy(x, y) = u(x) u(y) 4e?*”) Find the mean value of the function e**”. 


SECTION —IV 
8. a) Write and prove the properties of Auto correlation function 


b) Find the mean and Auto correlation function of the Random process X(t) = Acos(wt+ @) 
where A and w are constants, 9 is random variable uniformly distributed on the interval 
(0,2F). 

OR 
9. a) X (t) is a WSS process and Y (t) = A cos (@ct + 9) is a random process which is 
independent of X(t). Here, 6 is a uniform random variable over (— x, 7). If the auto- 
correlation function of X (t) is Rxx(t), Find the auto correlation of Z()=X(t)Y(t). 

b) For a stationary Ergodic process X(t) with the periodic components the Auto correlation 
is Rxx(t)= 36+4/(1+5t’). Find E[X(t)], E[X?(t)] and Power in the process X(t). 


SECTION — V 


10. a)Derive an expression for Auto correlation function of response of a linear system with 
random input. 


b) Write about power density spectrum of Response of Linear system 


OR 
be Find the power density spectrum of a random process whose autocorrelation Function is 


R(t) =A cos (at). 
XX 


b) State and prove properties of power spectral density of random process. 


/ 
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Time: 3 hours Max. Marks: 
PART- A (25 Marks) 
1. a) Define probability using the axiomatic approach. 
b) Show that P(AUB)=P(A)+P(B)-P(ANB). 
c) Define Probability density and distribution function. 
d) Define various types of transformation of Random variables. 
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e) Define point conditioning & interval conditioning distribution function. 


f) Show that var(X+Y) = var(x)+var(Y), if X&Y are independent random variables. 


g) Define wide sense stationary random processes. 
h) Prove that the ACF has maximum value at the origin i.e | Rxx(T) | = Rxx(0) 
1) Define spectrum Band width and RMS bandwidth. 


j) If the Auto correlation function of wide sense stationary X(t) is Rxx(t)=4+2e 


Find the area enclosed by the power spectrum density curve of X(t). 


Part-B (5*10=50 Marks) 


a) State and Prove Bayes’ theorem. 
b) Explain the the Mathematical model of experiment. 
OR 


-2|T| 


75 


(2 marks) 
(3 marks) 
(2 marks) 
(3 marks) 
(2 marks) 
(3 marks) 
(2 marks) 
(3 marks) 
(2 marks) 


(3 marks) 


a) An experiment consists of observing the sum of the outcomes when two fair dice are thrown. 
Find the probability that the sum is 7 and find the probability that the sum is greater than 10. 

b) In a factory there are 4 machines produce 10%,20%,30%,40% of an items respectively. The 
defective items produced by each machine are 5%,4%,3% and 2% respectively. Now an item is 
selected which is to be defective, what is the probability it being from the 2"! machine. And 


also write the statement of total probability theorem? 


a) The exponential density function given by 


fx(x) =(1/)e x>a 
=0 x<a Find the mean and variance. 
b) Define Moment Generating Function and write any two properties. 


OR 


Derive the Binomial density function and find mean & variance. 


6. a) State and prove the density function of sum of two random variables. 
b) The joint density function of two random variables X and Y is 


(x + y)? 
fee(V).= + Age ; —1<x<1land-3<y<3 
0; otherwise 
Find the variances of X and Y. 


OR 


7: a) Let Z=X+Y-C, where X and Y are independent random variables with variance 6°x, 6”y 
and C is constant. Find the variance of Z in terms of 07x, o’y and C. 
b) State and prove any three properties of joint characteristic function. 


8. a) Define Wide Sense Stationary Process and write it’s conditions. 
b) A random process is given as X(t) = At, where A is a uniformly distributed random 
variable on (0,2). Find whether X(t) is wide sense stationary or not. 
OR 


o: X(t) is a stationary random process with a mean of 3 and an auto correlation function of 6+5 
exp (-0.2 | a | ). Find the second central Moment of the random variable Y=Z-W, where ‘Z’ 
and ‘W’ are the samples of the random process at t=4 sec and t=8 sec respectively. 


10. a) Check the following power spectral density functions are valid or not 
-) COSBW@) «| —-(w—1)? 
i) “Sak ii) e 
b) Derive the relation between input PSD and output PSD of an LTI system 
OR 
11. Derive the relationship between cross-power spectral density and cross correlation function. 


Time: 3 hours 


1. 
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PART- A (25 Marks) 
a. Define subset and state its properties. 


b. Define sample space and classify the types of sample space. 

c. Define the expected value of Discrete and Continuous Random Variables. 

d. Derive the expression for the density function of Discrete Random variable. 

e. Define the statistical Independence of the Random variables. 

f. If E[X]=2, E[Y]=3, E[XY]=10, E[X*]=9, and E[Y7]=16 then find variance & 
Covariance of X&Y. 

g. Give the statement of ergodic theorem. 

h. Differentiate between Random Processes and Random variables with example 


i. Define Power Spectrum Density. 


j. Show that Sxx(-@) = Sxx(@). 1-e., Power spectrum density is even function of @. 


Part-B (5*10=50 Marks) 


In a box there are 100 resistors having resistance and tolerance values given in table. Let a resistor 
be selected from the box and assume that each resistor has the same likelihood of being chosen. 
Event A: Draw a 47Q resistor, Event B: Draw a resistor with 5% tolerance, Event C: Draw a 100Q 


resistor. Find the individual, joint and conditional probabilities. 


Resistance Tolerance Total 

(Q) 5% 10% 

22 10 14 24 

47 28 16 44 

100 24 8 32 

Total 62 38 100 
OR 


a)Two boxes are selected randomly. The first box contains 2 white balls and 3 black balls. The 


(2 marks) 
(3 marks) 
(2 marks) 
(3 marks) 
(2 marks) 


(3 marks) 
(2 marks) 
(3 marks) 
(2 marks) 
(3 marks) 


second box contains 3 white and 4 black balls. What is the probability of drawing a white ball? 


b) An aircraft is used to fire at a target. It will be successful if 2 or more bombs hit the target. If the 
aircraft fires 3 bombs and the probability of the bomb hitting the target is 0.4, then what is the 


probability that the target is hit? 


10. 


11. 


Derive the Poisson density function and find its mean & variance. 


OR 
If X is a discrete random variable with a Moment generating function of M,(v), find the Moment 
generating function of 
i) Y=aX+bii)Y=KX iii) Y=" 


a) State and explain the properties of joint density function 
b) The joint density function of random variables X and Y is 


_ (8xy; O<x<10<y<1 
fay Coy) = 0, otherwise 
Find f(y/x) and f(x/y) 


OR 


The input to a binary communication system is a RV X, takes on one of two values 0 and 1, with 
probabilities %4 and % respectively. Due to the errors caused by the channel noise, the output 
random variable Y, differs from the Input X occasionally. The behavior of them communication 
‘ii 3 iz 
X=1 


system is modeled by the conditional probabilities P( -) Wi and P (=) =a Find 


a) The probability for a transmitted message to be received as 0 
b) Probability that the transmitted message is al. If the received is a 1. 


Explain the following 
i) Stationary 
i) Ergodicity 
ii) Distribution & density functions of random processes 
OR 
a) Given the RP X(t) = A cos(Wot) + B sin (Wot) where @o is a constant, and A and B are 
uncorrelated Zero mean random variables having different density functions but the same 
variance o*. Show that X(t) is wide sense stationary. 
b) Define Covariance of the Random processes with any two properties. 


A stationery random process X(t) has spectral density Sxx(@)=25/ (w*+25) and an independent 


stationary process Y(t) has the spectral density Syy(@)= w*/ (w*+25). If X(t) and Y(t) are of zero 
mean, find the: 


a) PSD of Z(t=X(t) + Y(t) 
b) Cross spectral density of X(t) and Z(t) 
OR 
a) The input to an LTI system with impulse response h(t)= 6(t) + t*e~“. U(t) is a WSS process 
with mean of 3. Find the mean of the output of the system. 
b) Define Power Spectral density with three properties. 
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PART- A (25 Marks) 


1. a. Give the classical definition of Probability with example. (2 marks) 
b. Define Joint and Conditional Probability (3 marks) 
c. Define Moment generating function and Characteristic Function of a RV’s (2 marks) 
d. Find the variance of X for uniform density function. (3 marks) 
e. Give the statement of central limit theorem. (2 marks) 
f. The joint probability density function of X&Y is 
fxy(x%y) = c(2x+y); 0<x<2,0<y<3 
0; else 
Then find the value of constant c. (3 marks) 


g. Define the auto covariance & cross covariance functions of Random processes X(t).(2 marks) 
h. A stationary ergodic random processes has the Auto correlation function with the periodic 


components as Rxx(t) = 25 + 116d (3 marks) 
i. Give the statement of Wiener-Khinchin relation. (2 marks) 
j. If the Power spectrum density of x(t) is Sxx(@), find the PSD of < x(t). (3 marks) 


PART-B (5*10=50 Marks) 


2. Determine probabilities of system error and correct system transmission of symbols for an 
elementary binary communication system shown in below figure consisting of a transmitter that 
sends one of two possible symbols (a | or a 0) over a channel to a receiver. The channel 
occasionally causes errors to occur so that a ’1’ show up at the receiver as a ’0? And vice versa. 
Assume the symbols ‘1’ and ‘0’ are selected for a transmission as 0.6 and 0.4 respectively. 


P(B1)=0.6 


P(B2)=0.4 


OR 


10. a) A random process Y(t) has the power spectral density Syy(@)= 


11. 


6699 


In a binary communication system, the errors occur with a probability of “p”, In a block of “n” 
bits transmitted, what is the probability of receiving 

1) at the most | bit in error 

i1 at least 4 bits in error 


A random variable X has the distribution function 


12 2 
Fx(x)= >. Gap U(X —n) 
n=1 
Find the probability of a) P{-00o < X < 6.5} b)p{X>4} c) p{6< X <9} 


OR 
Let X be a Continuous random variable with density function 
f=  5+K  Osxx6 
0 otherwise 


Find the value of K and also find P{2 < X < 5} 


Let X and Y be the random variables defined as X=Cos@ and Y=Sin@ where 0 is a uniform 
random variable over (0, 272) 
a) Are X and Y Uncorrelated? 
b) Are X and Y Independent? 

OR 
a) Define and State the properties of joint cumulative distribution function of two random 
variables X and Y. 


b) A joint probability density function is fry(Xy ) = 0<x<60<y<4 


0 else where 
Find the expected value of the function g(X, Y)= (xy) 
' : : 6 sin (mT ) : . 
a) A Gaussian RP has an auto correlation function Rxx()}=—_ Determine a covariance 


matrix for the Random variable X(t) 
b) Derive the expression for cross correlation function between the input and output of a LTI 
system. 
OR 
a) Derive the mean and mean square value of response of LTI system. 
b)Discuss in detail about first order stationary random process. 


9 
w?+64 
Find i) The average power of the process 
ii) The Auto correlation function 
b) State the properties of power spectral density 

OR 
wW 


a) A random process has the power density spectrum Syy(@)=—, . Find the average power in 


6w2 
1+04 * 
the process. 


. ; : 2 - he 
b) Find the auto correlation function of the random process whose psd is oat 
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[. 


PART- A (25 Marks) 


a. Write the statement of multiplication theorem. (2 marks) 


b. Define Equally likely events, Exhaustive events and Mutually exclusive events. (3 marks) 


c. Define moments about origin and central moments. (2 marks) 
d. Write the properties of Gaussian density curve. (3 marks) 
e. Define correlation and covariance of two random variables X& Y. (2 marks) 
f. Define correlation coefficient with two properties. (3 marks) 
g. When two random processes X(t)& Y(t) are said to be independent. (2 marks) 


h. Define mean ergodic random processes & correlation ergodic Random processes. (3 marks) 
i. Write any two properties of Power Spectrum Density (2 marks) 


j. Define linear system and impulse response. (3 marks) 


PART-B (5*10=50 Marks) 


Let A and B are events in a sample space S. Show that if A and B are independent, then so are 
a) AandB b) A andBc)A andB 

OR 
In the experiment of tossing a die, all the even numbers are equally likely to appear and 
similarly the odd numbers. An odd number occurs thrice more frequently than an even number. 
Find the probability that 
a) An even number appears 
b) A prime number appears 
c) An odd numbers appears 


a) Verify the Characteristic function of a random variable is having its maximum magnitude at 
wo=0 and find its maximum value. 
b) Find the Moment generating function of exponential distribution? 
OR 
The probability density function of a random variable X is given by f(x) — for -3<x<6 ande 


qual to zero otherwise. Find the density function of Y=>(12-x) 


10. 


11. 


State and prove the central limit theorem 
OR 
Two random variables X and Y have zero mean and variance of = 16 and of = 36 
correlation coefficient is 0.5 determine the following 
i) The variance of the sum of X and Y 
il) The variance of the difference of X and Y 


The function of time Z(t) = X jcos@ot- X2sin@ot is a random process. If X; and Xzare 
independent Gaussian random variables, each with zero mean and variance o°, find E[Z]. E[Z’] 
and var(z). 

OR 
Briefly explain the distribution and density functions in the context of stationary and 
independent random processes. 


a)Find the cross correlation function corresponding to the cross power spectrum 


_ 6 
SxrO)"aT, 2)(3+4j w)2 


b) Write short notes on cross power density spectrum. 


OR 
a) Consider a random process X(t)=cos(wt + 8)where w is a real constant and is a uniform 


random variable in (0, 2/2). Find the average power in the process. 
b) Define and derive the expression for average power of Random process. 


28 KK 
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PART- A (25 Marks) 

a. What are the conditions for a function to be a random variable? (2 marks) 
b. Define Random variable and Write the classifications of Random variable (3 marks) 
c. Show that Var(kX)=k? var(X), here k is a constant. (2 marks) 
d. In an experiment when two dice are thrown simultaneously, find expected value of 

the sum of number of points on them. (3 marks) 
e. Define the joint Gaussian density function of two random variables. (2 marks) 
f. Define Marginal distribution & Density functions. (3 marks) 
g. Define the cross correlation function between two random processes X(t) & Y(t). (2 marks) 
h. Find the mean value of Response of a linear system. (3 marks) 
i. Define linear system. (2 marks) 
j. If X(t) & Y(t)are uncorrelated and have constant mean values X&Y then show that 

Sxx(@)= 2ILXY6(w) (3 marks) 


PART-B (5*10=50 Marks) 


a) An experiment consist of rolling a single die. Two events are defined as A = { 6 shows up}: and 
B= {2 or 5 shows up} 
i) Find P(A) and P(B) 
ii) Define a third event C so that P(C) = 1-P(A)-P(B) 
b) The six sides of a fair die are numbered from t to 6. The die is rolled 4 times. How many 
sequences of the four resulting numbers are possible? 
OR 
a) State and prove the total probability theorem? 
b) Explain about conditional probability 


a) Write short notes on Gaussian distribution and also find its mean? 
b) Consider that a fair coin is tossed 3 times, Let X be a random variable, defined as 
X= number of tails appeared, find the expected value of X. 
OR 

a) Find the characteristic function and first moment for 

fx(x)=(1/b)exp(-(x-a)/b) x>a 

=0 else 

b) Find the probability of getting a total of 5 or 11, when tossing a pair of fair dice. 


A certain binary system transmits two binary states X = +1 and X = -1 with equal probability. 
There are three possible states with the receiver, such as Y = +1, O and -1. The performance of 
the communication system is given as 

Pry = +1/X = +1) =0.2; 

P(Y= +1/X= -1) = 0.1; P(Y = 0/X = +1) = P(Y = 0/X = -1) = 0.05. Find 


10. 


11. 


a) P(Y =0) b) PX =+41/Y =+1) 
c) P(X =-1/Y =0). 

OR 
Two random variables X and Y have the joint pdf is 


fx y(XY)= x,y20 
0 elsewhere 


i. Evaluate A 

ii. Find the marginal pdf’s 

ili. Find the marginal pdf’s 

iv. Find the joint cdf 

v. Find the distribution functions and conditional cdf’s. 


Explain about the following random process 
(i) Mean ergodic process 
(ii) Correlation ergodic process 
(iii) Gaussian random process 


OR 
State and prove the auto correlation and cross correlation function properties. 


a) The power spectrum density function of a stationary random process is given by 
Sxx(@)= A, -K< w< K 
0, other wise 
Find the auto correlation function. 
b) Derive the expression for power spectrum density. 


OR 
a) Define and derive the expression for average cross power between two random process X(t) and 


Y(t). 


2 
b) Find the cross power spectral density for Rxx(=> sini€iwoT). 
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